This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch release/v1.3
in repository https://gitbox.apache.org/repos/asf/texera.git
The following commit(s) were added to refs/heads/release/v1.3 by this push:
new 503609751e feat(deploy, v1.3): replace MinIO with RustFS as the
default object store (#8550)
503609751e is described below
commit 503609751eadad3ef88dfd9b8488ae9f7df131ad
Author: Meng Wang <[email protected]>
AuthorDate: Thu Sep 17 03:35:04 2026 +0000
feat(deploy, v1.3): replace MinIO with RustFS as the default object store
(#8550)
### What changes were proposed in this PR?
Backport of #8542 to `release/v1.3`: a clean cherry-pick of its squash
commit, no adaptations — every added and removed line is identical to
#8542's. See #8542 for the change itself (MinIO → RustFS in the compose
files, the Helm chart, CI, tests and docs).
Why a `feat` goes to a release branch: MinIO removed its Docker Hub
images on 2026-09-11 (`minio/minio:RELEASE.2025-02-28T09-55-16Z` and
`minio/mc` now return 404), so a v1.3 built from this branch can neither
be deployed with its default object store nor pass its integration CI.
It also un-breaks the pre-merge backport check for every amber change:
`Backport Checks` runs `main`'s `build.yml`, which since #8542
provisions RustFS with `texera_rustfs`, against this branch's tree,
whose `storage.conf` still said `texera_minio`, so `aws s3api
create-bucket` fails with `InvalidAccessKeyId` (e.g. #8432).
v1.3 has not been released, so #8542's migration note (object-store
on-disk format and volume names change, no in-place upgrade) affects no
deployed release.
Source: 12aeeab5816b12880c5f58970460adf87b548d89
### Any related issues, documentation, discussions?
Backport of #8542.
### How was this PR tested?
The branch is verified to be exactly #8542's squash commit cherry-picked
onto `release/v1.3` (48 files, no conflicts; the added/removed lines
diff empty against #8542). Seven of those files were also changed on
`main` after the v1.3 cut (#7930, #8307, #7664, #8032, #6866, #8073);
#8542's hunks in them are self-contained MinIO→RustFS renames and
reference none of those later additions. Locally on this tree: `helm
dependency build` and `helm template` with the default and
`values-development.yaml` values render cleanly with no remaining
`minio` reference, and `helm lint` passes; `sbt
WorkflowCore/Test/compile FileService/Test/compile` succeed (the two
amber Scala changes are comment-only); `ruff check` and `ruff format
--check` at CI's pinned 0.14.7 pass on the changed `amber/` Python
files. Release-branch CI on this PR runs the full matrix, with the
integration jobs now provisioning RustFS.
### Was this PR authored or co-authored using generative AI tooling?
Yes. Generated-by: Claude Code (claude-fable-5-1)
Co-authored-by: Ian Maxon <[email protected]>
Co-authored-by: Claude Opus 5 <[email protected]>
---
.github/scripts/smoke-boot.sh | 4 +-
.github/workflows/build.yml | 81 +++++++++++-----------
amber/src/main/python/core/runnables/main_loop.py | 17 ++---
.../texera/web/service/LakekeeperClient.scala | 2 +-
.../texera/web/service/WorkflowService.scala | 2 +-
.../packaging/test_state_materialization_e2e.py | 2 +-
.../core/storage/iceberg/test_iceberg_document.py | 4 +-
.../python/core/storage/test_storage_config.py | 4 +-
.../pytexera/storage/test_large_binary_manager.py | 4 +-
bin/k8s/Chart.yaml | 8 +--
bin/k8s/templates/README.md | 2 +-
bin/k8s/templates/aws/s3-credentials-secret.yaml | 4 +-
bin/k8s/templates/base/_helpers.tpl | 12 ++--
.../base/external-names/external-names.yaml | 8 +--
bin/k8s/templates/base/gateway/gateway-routes.yaml | 10 +--
bin/k8s/templates/base/gateway/gateway.yaml | 12 ++--
.../base/lakekeeper/lakekeeper-init-job.yaml | 50 +++++++++++--
...io-persistence.yaml => rustfs-persistence.yaml} | 8 +--
bin/k8s/values-aws.yaml | 6 +-
bin/k8s/values-development.yaml | 52 +++++++++-----
bin/k8s/values.yaml | 79 ++++++++++++++-------
bin/local-dev/README.md | 4 +-
bin/local-dev/docker-compose.override.yml | 4 +-
bin/local-dev/main.sh | 36 +++++-----
bin/local-dev/tui.py | 2 +-
bin/single-node/.env | 10 +--
bin/single-node/README.md | 4 +-
bin/single-node/docker-compose.yml | 65 +++++++++--------
build.sbt | 3 +-
common/config/src/main/resources/storage.conf | 2 +-
common/workflow-core/build.sbt | 7 +-
.../texera/service/util/S3StorageClient.scala | 8 +--
.../apache/texera/common/tags/NonParallelTest.java | 2 +-
.../util/LakeFSStorageClientMtimeSpec.scala | 43 ++++++------
.../texera/service/util/RustFSContainer.scala | 80 +++++++++++++++++++++
.../texera/service/util/S3StorageClientSpec.scala | 4 +-
.../texera/service/util/S3StorageTestBase.scala | 21 +++---
.../guide-for-developers.md | 6 +-
docs/getting-started/installing-using-docker.md | 4 +-
docs/getting-started/run-on-kubernetes.md | 4 +-
docs/overview.md | 2 +-
file-service/build.sbt | 3 +-
file-service/src/main/resources/docker-compose.yml | 25 ++++---
.../{minio-config.yml => rustfs-config.yml} | 23 +++---
.../texera/service/util/StagedFileCleanupJob.scala | 34 +++++++--
.../org/apache/texera/service/MockLakeFS.scala | 60 ++++++++--------
.../resource/ModelDownloadResourceSpec.scala | 2 +-
.../service/user/model/model.service.spec.ts | 8 +--
48 files changed, 523 insertions(+), 314 deletions(-)
diff --git a/.github/scripts/smoke-boot.sh b/.github/scripts/smoke-boot.sh
index 9b98c67a79..1b56205447 100755
--- a/.github/scripts/smoke-boot.sh
+++ b/.github/scripts/smoke-boot.sh
@@ -43,8 +43,8 @@
# * TEXERA_HOME must point at the checkout root -- services resolve their
# config yaml from <TEXERA_HOME>/<service>/src/main/resources/...
# * the service's backing infra must already be up (postgres for every
service,
-# plus MinIO + LakeFS for file-service); the JVM connects via storage.conf
-# defaults (postgres/postgres @ localhost:5432, MinIO :9000, LakeFS :8000).
+# plus RustFS + LakeFS for file-service); the JVM connects via storage.conf
+# defaults (postgres/postgres @ localhost:5432, RustFS :9000, LakeFS
:8000).
set -euo pipefail
diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
index c0107997b3..f4c041a4fa 100644
--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -385,7 +385,7 @@ jobs:
# slow macOS runner without masking a real hang.
timeout-minutes: 25
strategy:
- # macOS provisions postgres / minio / lakekeeper natively because
+ # macOS provisions postgres / rustfs / lakekeeper natively because
# GitHub-hosted macOS runners have no Docker (and `services:`
# containers are Linux-only). Each docker-dependent step below
# branches on $RUNNER_OS inside its `run:` script: Linux keeps
@@ -508,30 +508,34 @@ jobs:
run: psql -h localhost -U postgres -v DB_NAME=texera_db_for_test_cases
-f sql/texera_ddl.sql
env:
PGPASSWORD: postgres
- - name: Start MinIO
- # Linux uses the pinned docker image; macOS uses brew's native
- # arm64 binary, backgrounded via nohup and logged to /tmp/minio.log
- # for post-mortem if the curl health check below fails. The brew
- # version drifts from the Linux pin, but the tests only touch
- # S3-protocol surface that has been stable across releases.
+ - name: Start RustFS
+ # Linux uses the pinned docker image; macOS downloads the
+ # same-version aarch64 zip that upstream publishes alongside it,
+ # backgrounded via nohup and logged to /tmp/rustfs.log for
+ # post-mortem if the curl health check below fails.
run: |
if [ "$RUNNER_OS" = "Linux" ]; then
- docker run -d --name minio --network host \
- -e MINIO_ROOT_USER=texera_minio \
- -e MINIO_ROOT_PASSWORD=password \
- minio/minio:RELEASE.2025-02-28T09-55-16Z server /data
+ docker run -d --name rustfs --network host \
+ -e RUSTFS_ACCESS_KEY=texera_rustfs \
+ -e RUSTFS_SECRET_KEY=password \
+ -e RUSTFS_REGION=us-west-2 \
+ rustfs/rustfs:1.0.0-rc.6
else
- brew install minio/stable/minio
- mkdir -p /tmp/minio-data
- MINIO_ROOT_USER=texera_minio MINIO_ROOT_PASSWORD=password \
- nohup minio server /tmp/minio-data > /tmp/minio.log 2>&1 &
+ curl -sSfL -o /tmp/rustfs.zip \
+
https://dl.rustfs.com/artifacts/rustfs/release/rustfs-macos-aarch64-v1.0.0-rc.6.zip
+ unzip -q -o /tmp/rustfs.zip -d /tmp/rustfs-bin
+ chmod +x /tmp/rustfs-bin/rustfs
+ mkdir -p /tmp/rustfs-data
+ RUSTFS_ACCESS_KEY=texera_rustfs RUSTFS_SECRET_KEY=password \
+ RUSTFS_REGION=us-west-2 RUSTFS_ADDRESS=:9000 \
+ nohup /tmp/rustfs-bin/rustfs /tmp/rustfs-data > /tmp/rustfs.log
2>&1 &
fi
for i in $(seq 1 15); do
- curl -sf http://localhost:9000/minio/health/live && break
- echo "Waiting for MinIO... (attempt $i)"
+ curl -sf http://localhost:9000/health && break
+ echo "Waiting for RustFS... (attempt $i)"
sleep 1
done
- curl -sf http://localhost:9000/minio/health/live
+ curl -sf http://localhost:9000/health
- name: Start Lakekeeper
# Linux uses the v0.11.0 docker image; macOS downloads the
# same-version aarch64-apple-darwin tarball that upstream
@@ -608,18 +612,14 @@ jobs:
LAKEKEEPER_BASE=${REST_URI%/catalog}
LAKEKEEPER_BASE=${LAKEKEEPER_BASE%/}
- # bucket creation runs through `mc`; on Linux we keep the
- # minio/mc image, on macOS we use the brew-installed native CLI
- # since docker is unavailable.
- if [ "$RUNNER_OS" = "Linux" ]; then
- docker run --rm --network host --entrypoint sh minio/mc -c \
- "mc alias set minio $S3_ENDPOINT $S3_USERNAME $S3_PASSWORD && \
- mc mb --ignore-existing minio/$S3_BUCKET"
- else
- brew install minio-mc
- mc alias set minio "$S3_ENDPOINT" "$S3_USERNAME" "$S3_PASSWORD"
- mc mb --ignore-existing "minio/$S3_BUCKET"
- fi
+ # Bucket creation goes through the AWS CLI, which is preinstalled on
+ # both the Linux and macOS runner images -- so unlike the rest of
this
+ # job it needs no $RUNNER_OS branch. CreateBucket on an existing
bucket
+ # is a no-op on RustFS, which keeps re-runs idempotent.
+ AWS_ACCESS_KEY_ID="$S3_USERNAME" \
+ AWS_SECRET_ACCESS_KEY="$S3_PASSWORD" \
+ AWS_DEFAULT_REGION="$S3_REGION" \
+ aws s3api create-bucket --bucket "$S3_BUCKET" --endpoint-url
"$S3_ENDPOINT"
curl -sf -X POST -H 'Content-Type: application/json' \
-d
'{"project-id":"00000000-0000-0000-0000-000000000000","project-name":"default"}'
\
"$LAKEKEEPER_BASE/management/v1/project" || true
@@ -1004,23 +1004,24 @@ jobs:
run: |
mkdir -p /tmp/dists
unzip -q ${{ matrix.service }}/target/universal/${{ matrix.service
}}-*.zip -d /tmp/dists/
- - name: Start MinIO
+ - name: Start RustFS
# file-service's boot creates its S3 bucket via S3StorageClient; only
# that service needs an object store, so the rest of the matrix skips
this.
if: ${{ matrix.object_store && steps.module_check.outputs.present ==
'true' }}
run: |
- docker run -d --name minio --network host \
- -e MINIO_ROOT_USER=texera_minio \
- -e MINIO_ROOT_PASSWORD=password \
- minio/minio:RELEASE.2025-02-28T09-55-16Z server /data
+ docker run -d --name rustfs --network host \
+ -e RUSTFS_ACCESS_KEY=texera_rustfs \
+ -e RUSTFS_SECRET_KEY=password \
+ -e RUSTFS_REGION=us-west-2 \
+ rustfs/rustfs:1.0.0-rc.6
for i in $(seq 1 15); do
- curl -sf http://localhost:9000/minio/health/live && break
- echo "Waiting for MinIO... (attempt $i)"; sleep 1
+ curl -sf http://localhost:9000/health && break
+ echo "Waiting for RustFS... (attempt $i)"; sleep 1
done
- curl -sf http://localhost:9000/minio/health/live
+ curl -sf http://localhost:9000/health
- name: Start LakeFS
# lakeFS keeps its metadata in the texera_lakefs postgres DB (created
- # above) and uses MinIO as its S3 blockstore. file-service's boot calls
+ # above) and uses RustFS as its S3 blockstore. file-service's boot
calls
# LakeFSStorageClient.healthCheck(), so this must be up for it to
reach a
# listening state. Config mirrors bin/single-node (compose + .env),
# adapted to CI creds (postgres/postgres @ localhost).
@@ -1032,7 +1033,7 @@ jobs:
-e LAKEFS_BLOCKSTORE_TYPE=s3 \
-e LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=true \
-e LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://localhost:9000 \
- -e LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=texera_minio \
+ -e LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=texera_rustfs \
-e LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=password \
-e LAKEFS_AUTH_ENCRYPT_SECRET_KEY=random_string_for_lakefs \
--entrypoint /bin/sh \
diff --git a/amber/src/main/python/core/runnables/main_loop.py
b/amber/src/main/python/core/runnables/main_loop.py
index 108078ba4f..6fcae0c26b 100644
--- a/amber/src/main/python/core/runnables/main_loop.py
+++ b/amber/src/main/python/core/runnables/main_loop.py
@@ -165,14 +165,15 @@ class MainLoop(StoppableQueueBlockingRunnable):
# read is deferred to here rather than done when the state arrives: at
# arrival time THIS worker's own materialization reader is still
# streaming its input, and in runs where this read overlapped that
- # reader, the reader failed with S3 "Access Denied" (MinIO's answer
- # for a deleted key) while iterating a lazily-pinned snapshot of a doc
- # that region re-execution drops and recreates. Removing the overlap
- # made those failures stop; that the overlap CAUSED them is the
- # working hypothesis, not a ruled-out fact -- a doc dropped under a
- # live reader would break the reader with or without this read -- so
- # treat a recurrence as new evidence. By EndChannel the reader has
- # finished, so the two never overlap.
+ # reader, the reader failed with S3 "Access Denied" (which was MinIO's
+ # answer for a deleted key; RustFS returns the standard NoSuchKey, so a
+ # recurrence will surface under that code instead) while iterating a
+ # lazily-pinned snapshot of a doc that region re-execution drops and
+ # recreates. Removing the overlap made those failures stop; that the
+ # overlap CAUSED them is the working hypothesis, not a ruled-out fact
+ # -- a doc dropped under a live reader would break the reader with or
+ # without this read -- so treat a recurrence as new evidence. By
+ # EndChannel the reader has finished, so the two never overlap.
if self._pending_loop_state is None:
return
pending = self._pending_loop_state
diff --git
a/amber/src/main/scala/org/apache/texera/web/service/LakekeeperClient.scala
b/amber/src/main/scala/org/apache/texera/web/service/LakekeeperClient.scala
index cd2d48669b..eebde38cf4 100644
--- a/amber/src/main/scala/org/apache/texera/web/service/LakekeeperClient.scala
+++ b/amber/src/main/scala/org/apache/texera/web/service/LakekeeperClient.scala
@@ -87,7 +87,7 @@ class LakekeeperClient(
/**
* Creates a warehouse backed by this deployment's own object store (the
Local flavor):
- * the storage profile points at the configured MinIO/S3 endpoint and
bucket, with the
+ * the storage profile points at the configured S3 endpoint and bucket,
with the
* platform's static credentials and STS off.
*
* @return the Lakekeeper-assigned warehouse id.
diff --git
a/amber/src/main/scala/org/apache/texera/web/service/WorkflowService.scala
b/amber/src/main/scala/org/apache/texera/web/service/WorkflowService.scala
index 9ea3e48c12..02394bee97 100644
--- a/amber/src/main/scala/org/apache/texera/web/service/WorkflowService.scala
+++ b/amber/src/main/scala/org/apache/texera/web/service/WorkflowService.scala
@@ -358,7 +358,7 @@ class WorkflowService(
* 2. Clears URI references from the execution registry
* 3. Safely clears all result and console message documents
* 4. Expires Iceberg snapshots for runtime statistics
- * 5. Deletes this execution's large binaries from MinIO
+ * 5. Deletes this execution's large binaries from the object store
*
* @param eid The execution identity to clean up resources for
*/
diff --git
a/amber/src/test/python/core/architecture/packaging/test_state_materialization_e2e.py
b/amber/src/test/python/core/architecture/packaging/test_state_materialization_e2e.py
index 84c626f58b..0dbfe92c3a 100644
---
a/amber/src/test/python/core/architecture/packaging/test_state_materialization_e2e.py
+++
b/amber/src/test/python/core/architecture/packaging/test_state_materialization_e2e.py
@@ -80,7 +80,7 @@ _WAREHOUSE_DIR =
tempfile.mkdtemp(prefix="texera-state-e2e-warehouse-")
@pytest.fixture(scope="module", autouse=True)
def sqlite_iceberg_catalog():
"""Inject a sqlite-backed SqlCatalog so the test runs without external
- iceberg infra (postgres/minio).
+ iceberg infra (postgres/rustfs).
Note: the other iceberg-backed tests (e.g. test_iceberg_document.py) use a
postgres/REST catalog to mirror production. This e2e deliberately diverges
diff --git
a/amber/src/test/python/core/storage/iceberg/test_iceberg_document.py
b/amber/src/test/python/core/storage/iceberg/test_iceberg_document.py
index 4e2df0f34c..7c410876cc 100644
--- a/amber/src/test/python/core/storage/iceberg/test_iceberg_document.py
+++ b/amber/src/test/python/core/storage/iceberg/test_iceberg_document.py
@@ -66,8 +66,8 @@ StorageConfig.initialize(
commit_batch_size=4096,
s3_endpoint="http://localhost:9000";,
s3_region="us-east-1",
- s3_auth_username="minioadmin",
- s3_auth_password="minioadmin",
+ s3_auth_username="rustfsadmin",
+ s3_auth_password="rustfsadmin",
s3_large_binaries_base_uri="s3://texera-large-binaries/objects/0/",
)
diff --git a/amber/src/test/python/core/storage/test_storage_config.py
b/amber/src/test/python/core/storage/test_storage_config.py
index 5a76d4ccda..270f266161 100644
--- a/amber/src/test/python/core/storage/test_storage_config.py
+++ b/amber/src/test/python/core/storage/test_storage_config.py
@@ -31,7 +31,7 @@ _INIT_KWARGS = dict(
table_state_namespace="state_ns",
directory_path="/data/iceberg",
commit_batch_size="4096",
- s3_endpoint="http://minio:9000";,
+ s3_endpoint="http://rustfs:9000";,
s3_region="us-west-2",
s3_auth_username="s3_user",
s3_auth_password="s3_pass",
@@ -74,7 +74,7 @@ class TestInitialize:
assert fresh_config.ICEBERG_TABLE_RESULT_NAMESPACE == "result_ns"
assert fresh_config.ICEBERG_TABLE_STATE_NAMESPACE == "state_ns"
assert fresh_config.ICEBERG_FILE_STORAGE_DIRECTORY_PATH ==
"/data/iceberg"
- assert fresh_config.S3_ENDPOINT == "http://minio:9000";
+ assert fresh_config.S3_ENDPOINT == "http://rustfs:9000";
assert fresh_config.S3_REGION == "us-west-2"
assert fresh_config.S3_AUTH_USERNAME == "s3_user"
assert fresh_config.S3_AUTH_PASSWORD == "s3_pass"
diff --git
a/amber/src/test/python/pytexera/storage/test_large_binary_manager.py
b/amber/src/test/python/pytexera/storage/test_large_binary_manager.py
index 7bc3d7f555..55fa7b2fb8 100644
--- a/amber/src/test/python/pytexera/storage/test_large_binary_manager.py
+++ b/amber/src/test/python/pytexera/storage/test_large_binary_manager.py
@@ -46,8 +46,8 @@ def _init_storage_config():
commit_batch_size=1000,
s3_endpoint="http://localhost:9000";,
s3_region="us-east-1",
- s3_auth_username="minioadmin",
- s3_auth_password="minioadmin",
+ s3_auth_username="rustfsadmin",
+ s3_auth_password="rustfsadmin",
s3_large_binaries_base_uri=TEST_BASE_URI,
)
diff --git a/bin/k8s/Chart.yaml b/bin/k8s/Chart.yaml
index 8cda7ee227..aab1fa538d 100644
--- a/bin/k8s/Chart.yaml
+++ b/bin/k8s/Chart.yaml
@@ -46,10 +46,10 @@ dependencies:
version: 16.5.6
repository: https://charts.bitnami.com/bitnami
- - name: minio
- version: 15.0.7
- repository: https://charts.bitnami.com/bitnami
- condition: minio.enabled
+ - name: rustfs
+ version: 1.0.0-rc.6
+ repository: https://charts.rustfs.com
+ condition: rustfs.enabled
- name: lakefs
version: 1.8.1
diff --git a/bin/k8s/templates/README.md b/bin/k8s/templates/README.md
index 7047a885d1..0ee1914c84 100644
--- a/bin/k8s/templates/README.md
+++ b/bin/k8s/templates/README.md
@@ -27,7 +27,7 @@ these subdirectories are purely organizational — they do not
change rendering.
| Folder | Contains | Renders when |
|--------|----------|--------------|
| `base/` | Resources every deployment needs: the Texera micro-service
Deployments/Services, the Envoy Gateway + routes, Postgres/LakeFS/Lakekeeper
wiring, the computing-unit pool, RBAC and namespaces. | Always. |
-| `on-prem/` | Resources only used by a self-hosted / local deployment, e.g.
the in-cluster MinIO persistence. | Gated on the relevant on-prem value (e.g.
`minio.enabled`). |
+| `on-prem/` | Resources only used by a self-hosted / local deployment, e.g.
the in-cluster RustFS persistence. | Gated on the relevant on-prem value (e.g.
`rustfs.enabled`). |
| `aws/` | Resources only used on AWS/EKS, e.g. the external-S3 credentials
Secret, the AWS NLB/EIP `EnvoyProxy`, and the autoscaler warm-pool placeholder.
| Gated so they render to nothing off AWS (empty by default). |
Within `base/`, templates are further grouped into one subfolder per
diff --git a/bin/k8s/templates/aws/s3-credentials-secret.yaml
b/bin/k8s/templates/aws/s3-credentials-secret.yaml
index 8506115167..74acc4878a 100644
--- a/bin/k8s/templates/aws/s3-credentials-secret.yaml
+++ b/bin/k8s/templates/aws/s3-credentials-secret.yaml
@@ -17,8 +17,8 @@
# Credentials Secret for an external S3 store. Rendered only when an external
# endpoint is configured (storage.s3.endpoint) and the deployer has not
supplied
-# their own Secret (storage.s3.existingSecret). On the default in-cluster MinIO
-# install this renders nothing -- the services use MinIO's own Secret instead.
+# their own Secret (storage.s3.existingSecret). On the default in-cluster
RustFS
+# install this renders nothing -- the services use RustFS's own Secret instead.
{{- if and .Values.storage.s3.endpoint (not .Values.storage.s3.existingSecret)
}}
apiVersion: v1
kind: Secret
diff --git a/bin/k8s/templates/base/_helpers.tpl
b/bin/k8s/templates/base/_helpers.tpl
index e044b7285a..6955d53980 100644
--- a/bin/k8s/templates/base/_helpers.tpl
+++ b/bin/k8s/templates/base/_helpers.tpl
@@ -23,8 +23,8 @@ Object-storage (S3) resolution helpers.
When storage.s3.endpoint is set the services talk to that external
S3-compatible store (credentials come from storage.s3.existingSecret, or a
chart-generated "<release>-s3-credentials" Secret). When it is empty the
-services fall back to the in-cluster MinIO Service and its auto-generated
-"<release>-minio" Secret, so the default install is unchanged.
+services fall back to the in-cluster RustFS Service and its auto-generated
+"<release>-rustfs-secret" Secret, so the default install is unchanged.
*/}}
{{/* S3 endpoint URL. */}}
@@ -32,7 +32,7 @@ services fall back to the in-cluster MinIO Service and its
auto-generated
{{- if .Values.storage.s3.endpoint -}}
{{- .Values.storage.s3.endpoint -}}
{{- else -}}
-{{- printf "http://%s-minio:9000"; .Release.Name -}}
+{{- printf "http://%s-rustfs-svc:9000"; .Release.Name -}}
{{- end -}}
{{- end -}}
@@ -41,16 +41,16 @@ services fall back to the in-cluster MinIO Service and its
auto-generated
{{- if .Values.storage.s3.endpoint -}}
{{- .Values.storage.s3.existingSecret | default (printf "%s-s3-credentials"
.Release.Name) -}}
{{- else -}}
-{{- printf "%s-minio" .Release.Name -}}
+{{- printf "%s-rustfs-secret" .Release.Name -}}
{{- end -}}
{{- end -}}
{{/* Secret data key for the S3 access key id. */}}
{{- define "texera.s3.accessKeyIdKey" -}}
-{{- if .Values.storage.s3.endpoint -}}access-key-id{{- else -}}root-user{{-
end -}}
+{{- if .Values.storage.s3.endpoint -}}access-key-id{{- else
-}}RUSTFS_ACCESS_KEY{{- end -}}
{{- end -}}
{{/* Secret data key for the S3 secret access key. */}}
{{- define "texera.s3.secretAccessKeyKey" -}}
-{{- if .Values.storage.s3.endpoint -}}secret-access-key{{- else
-}}root-password{{- end -}}
+{{- if .Values.storage.s3.endpoint -}}secret-access-key{{- else
-}}RUSTFS_SECRET_KEY{{- end -}}
{{- end -}}
diff --git a/bin/k8s/templates/base/external-names/external-names.yaml
b/bin/k8s/templates/base/external-names/external-names.yaml
index 6879b8d7f7..d0a264290b 100644
--- a/bin/k8s/templates/base/external-names/external-names.yaml
+++ b/bin/k8s/templates/base/external-names/external-names.yaml
@@ -73,14 +73,14 @@ to access services in the main namespace using the same
service names.
"externalName" (printf "%s-svc.%s.svc.cluster.local" .Values.webserver.name
$namespace)
) | nindent 0 }}
-{{- if .Values.minio.enabled }}
+{{- if .Values.rustfs.enabled }}
---
-{{/* MinIO ExternalName -- only when the in-cluster MinIO is enabled; with an
+{{/* RustFS ExternalName -- only when the in-cluster RustFS is enabled; with an
external S3 store the CU pods reach it directly via STORAGE_S3_ENDPOINT.
*/}}
{{- include "external-name-service" (dict
- "name" (printf "%s-minio" .Release.Name)
+ "name" (printf "%s-rustfs-svc" .Release.Name)
"namespace" $workflowComputingUnitPoolNamespace
- "externalName" (printf "%s-minio.%s.svc.cluster.local" .Release.Name
$namespace)
+ "externalName" (printf "%s-rustfs-svc.%s.svc.cluster.local" .Release.Name
$namespace)
) | nindent 0 }}
{{- end }}
diff --git a/bin/k8s/templates/base/gateway/gateway-routes.yaml
b/bin/k8s/templates/base/gateway/gateway-routes.yaml
index f07a3157a4..8ca07e6309 100644
--- a/bin/k8s/templates/base/gateway/gateway-routes.yaml
+++ b/bin/k8s/templates/base/gateway/gateway-routes.yaml
@@ -167,24 +167,24 @@ spec:
port: 3001
{{- end }}
---
-# MinIO Route
-{{- if .Values.minio.gateway.enabled }}
+# RustFS Route
+{{- if .Values.rustfs.gateway.enabled }}
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
- name: texera-minio-route
+ name: texera-rustfs-route
namespace: {{ .Release.Namespace }}
spec:
parentRefs:
- name: {{ .Release.Name }}-gateway
hostnames:
- - {{ .Values.minio.gateway.hostname }}
+ - {{ .Values.rustfs.gateway.hostname }}
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- - name: {{ .Release.Name }}-minio
+ - name: {{ .Release.Name }}-rustfs-svc
port: 9000
{{- end }}
diff --git a/bin/k8s/templates/base/gateway/gateway.yaml
b/bin/k8s/templates/base/gateway/gateway.yaml
index 868439a673..dc79e39d6d 100644
--- a/bin/k8s/templates/base/gateway/gateway.yaml
+++ b/bin/k8s/templates/base/gateway/gateway.yaml
@@ -59,23 +59,23 @@ spec:
mode: Terminate
certificateRefs:
- name: {{ (index .Values "gatewayConfig" "tlsSecretName") | default
(printf "%s-cert" .Release.Name) }}
- {{- if .Values.minio.gateway.enabled }}
- - name: minio-http
+ {{- if .Values.rustfs.gateway.enabled }}
+ - name: rustfs-http
protocol: HTTP
port: 80
- hostname: {{ .Values.minio.gateway.hostname }}
+ hostname: {{ .Values.rustfs.gateway.hostname }}
allowedRoutes:
namespaces:
from: Same
- - name: minio-https
+ - name: rustfs-https
protocol: HTTPS
port: 443
- hostname: {{ .Values.minio.gateway.hostname }}
+ hostname: {{ .Values.rustfs.gateway.hostname }}
allowedRoutes:
namespaces:
from: Same
tls:
mode: Terminate
certificateRefs:
- - name: {{ .Values.minio.gateway.tlsSecretName | default (printf
"%s-minio-tls" .Release.Name) }}
+ - name: {{ .Values.rustfs.gateway.tlsSecretName | default (printf
"%s-rustfs-tls" .Release.Name) }}
{{- end }}
diff --git a/bin/k8s/templates/base/lakekeeper/lakekeeper-init-job.yaml
b/bin/k8s/templates/base/lakekeeper/lakekeeper-init-job.yaml
index d1adafcd92..3344c4effe 100644
--- a/bin/k8s/templates/base/lakekeeper/lakekeeper-init-job.yaml
+++ b/bin/k8s/templates/base/lakekeeper/lakekeeper-init-job.yaml
@@ -31,6 +31,48 @@ spec:
name: {{ .Release.Name }}-lakekeeper-init
spec:
restartPolicy: Never
+{{- if .Values.lakekeeperInit.createBucket }}
+ # Creates the Iceberg warehouse bucket before the warehouse is
registered.
+ # `rc` is RustFS's S3 client; using its image keeps the client out of the
+ # main container, which would otherwise have to fetch a binary at
runtime.
+ initContainers:
+ - name: create-bucket
+ image: {{ .Values.lakekeeperInit.bucketClientImage }}
+ env:
+ - name: STORAGE_S3_ENDPOINT
+ value: {{ include "texera.s3.endpoint" . }}
+ - name: STORAGE_S3_AUTH_USERNAME
+ valueFrom:
+ secretKeyRef:
+ name: {{ include "texera.s3.secretName" . }}
+ key: {{ include "texera.s3.accessKeyIdKey" . }}
+ - name: STORAGE_S3_AUTH_PASSWORD
+ valueFrom:
+ secretKeyRef:
+ name: {{ include "texera.s3.secretName" . }}
+ key: {{ include "texera.s3.secretAccessKeyKey" . }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET
+ value: {{ .Values.lakekeeperInit.warehouse.s3Bucket | quote }}
+ - name: STORAGE_ICEBERG_CATALOG_REST_REGION
+ value: {{ .Values.lakekeeperInit.warehouse.region | quote }}
+ command:
+ - /bin/sh
+ - -c
+ - |
+ set -e
+
+ echo "Waiting for the object store at ${STORAGE_S3_ENDPOINT}..."
+ until rc alias set local "${STORAGE_S3_ENDPOINT}" \
+ "${STORAGE_S3_AUTH_USERNAME}"
"${STORAGE_S3_AUTH_PASSWORD}" \
+ --region "${STORAGE_ICEBERG_CATALOG_REST_REGION}" \
+ --bucket-lookup path > /dev/null 2>&1; do
+ sleep 3
+ done
+
+ echo "Initializing object-store bucket
'${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}'..."
+ rc bucket create --ignore-existing
"local/${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}"
+ echo "Bucket '${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}' is
ready."
+{{- end }}
containers:
- name: lakekeeper-init
image: alpine:3.19
@@ -87,13 +129,7 @@ spec:
done
{{- if .Values.lakekeeperInit.createBucket }}
- echo "Step 1: Initializing object-store bucket
'${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}'..."
- apk add --no-cache wget
- wget -q https://dl.min.io/client/mc/release/linux-amd64/mc -O
/usr/local/bin/mc
- chmod +x /usr/local/bin/mc
- mc alias set minio "${STORAGE_S3_ENDPOINT}"
"${STORAGE_S3_AUTH_USERNAME}" "${STORAGE_S3_AUTH_PASSWORD}"
- mc mb --ignore-existing
minio/${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}
- echo "Bucket '${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}' is
ready."
+ echo "Step 1: object-store bucket
'${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}' was created by the create-bucket
init container."
{{- else }}
echo "Step 1: skipping bucket creation (createBucket=false;
using pre-existing external S3 bucket
'${STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET}')."
{{- end }}
diff --git a/bin/k8s/templates/on-prem/minio-persistence.yaml
b/bin/k8s/templates/on-prem/rustfs-persistence.yaml
similarity index 91%
rename from bin/k8s/templates/on-prem/minio-persistence.yaml
rename to bin/k8s/templates/on-prem/rustfs-persistence.yaml
index 4073c32582..dd0ce7cac8 100644
--- a/bin/k8s/templates/on-prem/minio-persistence.yaml
+++ b/bin/k8s/templates/on-prem/rustfs-persistence.yaml
@@ -17,11 +17,11 @@
{{/* Define storage path configuration, please change it to your own path and
make sure the path exists with the right permission*/}}
{{/* This path only works for local-path storage class */}}
-{{- $hostBasePath := .Values.persistence.minioHostLocalPath }}
+{{- $hostBasePath := .Values.persistence.rustfsHostLocalPath }}
-{{- if and .Values.minio.enabled .Values.minio.persistence.enabled }}
-{{- $name := "minio" }}
-{{- $persistence := .Values.minio.persistence }}
+{{- if and .Values.rustfs.enabled .Values.rustfs.persistence.enabled }}
+{{- $name := "rustfs" }}
+{{- $persistence := .Values.rustfs.persistence }}
{{- $volumeName := printf "%s-data-pv" $name }}
{{- $claimName := printf "%s-data-pvc" $name }}
{{- $storageClass := $persistence.storageClass | default "local-path" }}
diff --git a/bin/k8s/values-aws.yaml b/bin/k8s/values-aws.yaml
index 8d1b0a3698..82eabefa93 100644
--- a/bin/k8s/values-aws.yaml
+++ b/bin/k8s/values-aws.yaml
@@ -17,7 +17,7 @@
# ---------------------------------------------------------------------------
# Example overlay for running Texera against an external S3 store (e.g. AWS S3)
-# instead of the bundled in-cluster MinIO.
+# instead of the bundled in-cluster RustFS.
#
# helm install texera bin/k8s -f bin/k8s/values-aws.yaml
#
@@ -25,8 +25,8 @@
# own. The buckets must already exist (the init job does not create them).
# ---------------------------------------------------------------------------
-# Turn off the bundled MinIO object store; the services talk to external S3.
-minio:
+# Turn off the bundled RustFS object store; the services talk to external S3.
+rustfs:
enabled: false
persistence:
enabled: false
diff --git a/bin/k8s/values-development.yaml b/bin/k8s/values-development.yaml
index 5a8dc899e2..bddb3beeaa 100644
--- a/bin/k8s/values-development.yaml
+++ b/bin/k8s/values-development.yaml
@@ -22,7 +22,7 @@ texera:
imageTag: latest
global:
- # Required by Bitnami sub-charts (postgresql, minio) to allow custom images
+ # Required by the Bitnami postgresql sub-chart to allow custom images
security:
allowInsecureImages: true
@@ -34,10 +34,10 @@ global:
# - false: PVCs will remain after uninstall to preserve the data
persistence:
removeAfterUninstall: true
- minioHostLocalPath: ""
+ rustfsHostLocalPath: ""
postgresqlHostLocalPath: ""
-# Part 1: the configuration of Postgres, Minio and LakeFS
+# Part 1: the configuration of Postgres, RustFS and LakeFS
postgresql:
image:
repository: groonga/pgroonga
@@ -68,34 +68,50 @@ postgresql:
initdb:
scriptsConfigMap: "postgresql-init-script"
-minio:
- mode: standalone
+rustfs:
+ replicaCount: 1
+ drivesPerNode: 1
+ mode:
+ standalone:
+ enabled: true
+ existingClaim:
+ dataClaim: "rustfs-data-pvc"
+ distributed:
+ enabled: false
image:
- repository: bitnamilegacy/minio
- tag: 2025.3.12-debian-12-r0
+ rustfs:
+ repository: rustfs/rustfs
+ tag: 1.0.0-rc.6
resources:
requests:
memory: "256Mi"
limits:
memory: "256Mi"
+ ingress:
+ enabled: false
gateway:
enabled: false
- hostname: "" # the url for the minio, e.g. "minio.example.com"
- tlsSecretName: "" # e.g. "minio-tls-secret"
- auth:
- rootUser: texera_minio
- rootPassword: password
+ hostname: "" # the url for the object store, e.g. "rustfs.example.com"
+ tlsSecretName: "" # e.g. "rustfs-tls-secret"
+ secret:
+ rustfs:
+ access_key: texera_rustfs
+ secret_key: password
+ config:
+ rustfs:
+ region: us-west-2
+ obs_log_directory: ""
+ log_level: warn
service:
- # In production, use ClusterIP to avoid exposing the minio to the internet
+ # In production, use ClusterIP to avoid exposing the object store to the
internet
# type: ClusterIP
type: NodePort
- nodePorts:
- api: 31000
+ endpoint:
+ nodePort: 31000
persistence:
enabled: true
size: 20Gi
storageClass: local-path
- existingClaim: "minio-data-pvc"
lakefs:
secrets:
@@ -113,12 +129,12 @@ lakefs:
blockstore:
type: s3
s3:
- endpoint: http://texera-minio:9000
+ endpoint: http://texera-rustfs-svc:9000
pre_signed_expiry: 15m
pre_signed_endpoint: http://localhost:31000
force_path_style: true
credentials:
- access_key_id: texera_minio
+ access_key_id: texera_rustfs
secret_access_key: password
# Part2: configurations of Texera-related micro services
diff --git a/bin/k8s/values.yaml b/bin/k8s/values.yaml
index 7ab0ee5ae8..11b102b728 100644
--- a/bin/k8s/values.yaml
+++ b/bin/k8s/values.yaml
@@ -22,7 +22,7 @@ texera:
imageTag: 1.3.0-incubating
global:
- # Required by Bitnami sub-charts (postgresql, minio) to allow custom images
+ # Required by the Bitnami postgresql sub-chart to allow custom images
security:
allowInsecureImages: true
@@ -34,10 +34,10 @@ global:
# - false: PVCs will remain after uninstall to preserve the data
persistence:
removeAfterUninstall: true
- minioHostLocalPath: ""
+ rustfsHostLocalPath: ""
postgresqlHostLocalPath: ""
-# Part 1: the configuration of Postgres, Minio and LakeFS
+# Part 1: the configuration of Postgres, RustFS and LakeFS
postgresql:
image:
repository: groonga/pgroonga
@@ -66,47 +66,74 @@ postgresql:
scriptsConfigMap: "postgresql-init-script"
# Object storage (S3) used by the Texera services. Leave storage.s3.endpoint
-# empty to use the in-cluster MinIO (the `minio:` block below). Set it to an
+# empty to use the in-cluster RustFS (the `rustfs:` block below). Set it to an
# S3-compatible endpoint URL -- together with region and credentials -- to
# point the services at an external store (e.g. AWS S3) instead.
storage:
s3:
- endpoint: "" # "" => in-cluster MinIO; otherwise an S3 endpoint
URL
+ endpoint: "" # "" => in-cluster RustFS; otherwise an S3
endpoint URL
region: "us-west-2"
existingSecret: "" # existing Secret with access-key-id /
secret-access-key; "" => chart creates one
accessKeyId: ""
secretAccessKey: ""
-minio:
- # Set to false to disable the in-cluster MinIO and point the services at an
+rustfs:
+ # Set to false to disable the in-cluster RustFS and point the services at an
# external S3 store instead (configure storage.s3 above and the lakefs/
# lakekeeperInit blocks below). See values-aws.yaml for a complete example.
enabled: true
- mode: standalone
+ # Single-node object store, matching the previous standalone MinIO. The
+ # upstream chart defaults to a 4-replica distributed cluster; Texera's
+ # in-cluster store is a convenience for single-node installs, so keep it to
+ # one pod and one drive.
+ replicaCount: 1
+ drivesPerNode: 1
+ mode:
+ standalone:
+ enabled: true
+ existingClaim:
+ dataClaim: "rustfs-data-pvc"
+ distributed:
+ enabled: false
image:
- repository: bitnamilegacy/minio
- tag: 2025.3.12-debian-12-r0
+ rustfs:
+ repository: rustfs/rustfs
+ tag: 1.0.0-rc.6
+ # Texera fronts the object store with Envoy Gateway (the `gateway` block
+ # below), not with an Ingress -- the upstream chart turns one on by default.
+ ingress:
+ enabled: false
gateway:
enabled: false
- hostname: "" # the url for the minio, e.g. "minio.example.com"
- tlsSecretName: "" # e.g. "minio-tls-secret"
- auth:
- rootUser: texera_minio
- rootPassword: password
+ hostname: "" # the url for the object store, e.g. "rustfs.example.com"
+ tlsSecretName: "" # e.g. "rustfs-tls-secret"
+ secret:
+ rustfs:
+ access_key: texera_rustfs
+ secret_key: password
+ config:
+ rustfs:
+ # Must match storage.s3.region: the region is part of the SigV4 scope.
+ region: us-west-2
+ # Empty => log to stdout (`kubectl logs`) instead of a log PVC.
+ obs_log_directory: ""
+ log_level: warn
service:
- # In production, use ClusterIP to avoid exposing the minio to the internet
+ # In production, use ClusterIP to avoid exposing the object store to the
internet
# type: ClusterIP
type: NodePort
- nodePorts:
- api: 31000
+ endpoint:
+ nodePort: 31000
+ # The chart's own PVC template is unused here: Texera provisions the claim
+ # itself (templates/on-prem/rustfs-persistence.yaml) so that the local-path
+ # host directory stays under persistence.rustfsHostLocalPath.
persistence:
enabled: true
size: 20Gi
storageClass: local-path
- existingClaim: "minio-data-pvc"
lakefs:
- # The blockstore below points LakeFS at the in-cluster MinIO. To use an
+ # The blockstore below points LakeFS at the in-cluster RustFS. To use an
# external S3 store, override lakefsConfig (region-only blockstore) and
inject
# the S3 credentials via extraEnvVars -- see values-aws.yaml for an example.
secrets:
@@ -124,12 +151,12 @@ lakefs:
blockstore:
type: s3
s3:
- endpoint: http://texera-minio:9000
+ endpoint: http://texera-rustfs-svc:9000
pre_signed_expiry: 15m
pre_signed_endpoint: http://localhost:31000
force_path_style: true
credentials:
- access_key_id: texera_minio
+ access_key_id: texera_rustfs
secret_access_key: password
lakekeeper:
@@ -156,10 +183,14 @@ lakekeeper:
lakekeeperInit:
enabled: true
- # Create the object-store bucket from the init job (true for in-cluster
MinIO).
+ # Create the object-store bucket from the init job (true for in-cluster
RustFS).
# Set false when pointing at a pre-existing external bucket (e.g. AWS S3)
whose
# IAM principal can't CreateBucket.
createBucket: true
+ # S3 client image used by the create-bucket init container. `rc` is RustFS's
+ # client; any image whose entrypoint is an `mc`-style client with
+ # `alias set` / `bucket create` would do.
+ bucketClientImage: rustfs/rc:v0.1.35
defaultProject:
id: "00000000-0000-0000-0000-000000000000"
name: default
@@ -167,7 +198,7 @@ lakekeeperInit:
name: texera
region: us-west-2
s3Bucket: texera-iceberg
- # Lakekeeper S3 storage-profile flavor: "s3-compat" (MinIO/other; adds
+ # Lakekeeper S3 storage-profile flavor: "s3-compat" (RustFS/other; adds
# endpoint + path-style-access) or "aws" (real AWS S3; omits both).
flavor: s3-compat
# Object-key prefix for this warehouse. Must stay non-empty: a warehouse
at the
diff --git a/bin/local-dev/README.md b/bin/local-dev/README.md
index 83a889a749..ab84a9fbf6 100644
--- a/bin/local-dev/README.md
+++ b/bin/local-dev/README.md
@@ -42,7 +42,7 @@ platform — there is nothing to configure:
| Concern | macOS | Linux |
| --- | --- | --- |
-| Host LAN IP (the MinIO endpoint) | `route get default`, `ipconfig getifaddr`
| `ip route show default`, `ip -4 addr show scope global` |
+| Host LAN IP (the RustFS endpoint) | `route get default`, `ipconfig
getifaddr` | `ip route show default`, `ip -4 addr show scope global` |
| Artifact mtime (`watch`'s ARTIFACT MTIME column) | BSD `stat -f` | GNU `stat
-c` |
| Port → PID | `lsof` | `lsof`, falling back to `ss` |
@@ -83,7 +83,7 @@ bin/local-dev/
├── main.sh shell engine — sbt builds, service
lifecycle, port checks
├── tui.py Textual dashboard surfaced by
`bin/local-dev.sh -i`
├── docker-compose.override.yml overlay on top of
bin/single-node/docker-compose.yml
-│ (host-LAN-IP MinIO endpoint, Lakekeeper
warehouse, etc.)
+│ (host-LAN-IP RustFS endpoint, Lakekeeper
warehouse, etc.)
└── tests/
├── test_local_dev_sh.sh bash smoke: license header, syntax, version,
--help,
│ error-on-bad-input, regression guards
diff --git a/bin/local-dev/docker-compose.override.yml
b/bin/local-dev/docker-compose.override.yml
index 3d51fd1cf6..3b053a8570 100644
--- a/bin/local-dev/docker-compose.override.yml
+++ b/bin/local-dev/docker-compose.override.yml
@@ -38,11 +38,11 @@ services:
ports:
- "4000:4000"
- # The default .env hardcodes STORAGE_S3_ENDPOINT=http://texera-minio:9000
+ # The default .env hardcodes STORAGE_S3_ENDPOINT=http://texera-rustfs:9000
# — only routable inside the docker compose network. lakekeeper-init bakes
# that string into the warehouse's storage profile, which Lakekeeper then
# both dials itself (for s3 ops) AND returns to clients. The host-native
- # JVMs in local-dev mode can't resolve `texera-minio`.
+ # JVMs in local-dev mode can't resolve `texera-rustfs`.
#
# Use whatever STORAGE_S3_ENDPOINT the parent process exported. The
# wrapper script (bin/local-dev.sh) detects the host's LAN IP at startup
diff --git a/bin/local-dev/main.sh b/bin/local-dev/main.sh
index 10d0dea866..9053834acb 100755
--- a/bin/local-dev/main.sh
+++ b/bin/local-dev/main.sh
@@ -110,7 +110,7 @@
# agent-service :3001 Bun --watch (cd agent-service && bun
run dev)
# frontend :4200 ng serve via cd frontend && yarn
start
#
-# Docker infra (postgres / minio / lakefs / lakekeeper / litellm / jupyter) IS
managed
+# Docker infra (postgres / rustfs / lakefs / lakekeeper / litellm / jupyter)
IS managed
# here: `up` brings it up via `docker compose` (project texera-local-dev) and
# `down` tears down any docker targets. The script warns if expected ports are
# unreachable. Before any sbt build the postgres schema is reconciled: a fresh
@@ -582,12 +582,12 @@ elif [[ -s "$HOME/.volta/load.sh" ]]; then
fi
# --------- runtime env for backend ---------
-# Detect the host's primary LAN IP so we can use it as the MinIO endpoint.
+# Detect the host's primary LAN IP so we can use it as the RustFS endpoint.
# It has to be the same string from both directions:
# • host-native JVMs need it to reach localhost-published port 9000
# • the lakekeeper container needs it to do server-side S3 ops (validation,
# compaction) AND to return URLs to clients that *they* can reach
-# `localhost` only works for the host. `texera-minio` only works inside the
+# `localhost` only works for the host. `texera-rustfs` only works inside the
# docker network. The host's LAN IP works from BOTH (host loopback for the
# host, docker NAT'd out-and-back for the container).
#
@@ -630,7 +630,7 @@ _detect_host_lan_ip_linux() {
# 2. Scan every global IPv4, skipping the interfaces that would defeat the
# purpose of this address. A container bridge (docker0, br-*, veth*) is
# reachable from the host but not from inside another container's
- # network namespace the way MinIO needs; an overlay/VPN address
+ # network namespace the way RustFS needs; an overlay/VPN address
# (tailscale, zerotier) is not reachable from the docker bridge at all.
while read -r idx dev fam cidr _rest; do
[[ "$fam" == "inet" ]] || continue
@@ -654,7 +654,7 @@ _detect_host_lan_ip() {
}
# Lazy resolver — called from subcommands that actually need to publish a
# host-reachable S3 endpoint (cmd_up, cmd_auto). Subcommands like
-# `version`, `status`, `--help`, or `-i` don't talk to MinIO and shouldn't
+# `version`, `status`, `--help`, or `-i` don't talk to RustFS and shouldn't
# refuse to run just because the laptop is offline.
_require_host_lan_ip() {
[[ -n "${HOST_LAN_IP:-}" ]] && return 0
@@ -669,7 +669,7 @@ _require_host_lan_ip() {
bridge_note=" outside the container bridges" ;;
esac
echo "FATAL: could not detect a host LAN IP." >&2
- echo " MinIO needs an address reachable from both docker
(lakekeeper" >&2
+ echo " RustFS needs an address reachable from both docker
(lakekeeper" >&2
echo " does S3 ops) and the host (JVMs read signed URLs back);
none" >&2
echo " of $probes offered a non-loopback IPv4${bridge_note}." >&2
echo " Connect to a network or export HOST_LAN_IP=<your-IP>
explicitly." >&2
@@ -685,9 +685,9 @@ export
STORAGE_JDBC_URL="${STORAGE_JDBC_URL:-jdbc:postgresql://localhost:5432/te
export STORAGE_JDBC_USERNAME="${STORAGE_JDBC_USERNAME:-texera}"
export STORAGE_JDBC_PASSWORD="${STORAGE_JDBC_PASSWORD:-password}"
# STORAGE_S3_ENDPOINT is set lazily by _require_host_lan_ip — only the
-# subcommands that actually touch MinIO (infra_up + cmd_up + cmd_auto)
+# subcommands that actually touch RustFS (infra_up + cmd_up + cmd_auto)
# trigger that detection, so `version` / `status` / `-i` work offline.
-export STORAGE_S3_AUTH_USERNAME="${STORAGE_S3_AUTH_USERNAME:-texera_minio}"
+export STORAGE_S3_AUTH_USERNAME="${STORAGE_S3_AUTH_USERNAME:-texera_rustfs}"
export STORAGE_S3_AUTH_PASSWORD="${STORAGE_S3_AUTH_PASSWORD:-password}"
export STORAGE_S3_REGION="${STORAGE_S3_REGION:-us-west-2}"
export STORAGE_ICEBERG_CATALOG_TYPE="${STORAGE_ICEBERG_CATALOG_TYPE:-rest}"
@@ -874,7 +874,7 @@ _sbt_transitive_src_dirs() {
# --------- service catalog ---------
SERVICES=(
postgres
- minio
+ rustfs
lakefs
lakekeeper
litellm
@@ -899,7 +899,7 @@ SERVICES=(
# batch through infra_up/infra_down because `docker compose up -d` and
# `docker compose down` operate at the project level.
amap_set SVC_TYPE postgres docker; amap_set SVC_PORT postgres 5432;
amap_set SVC_CWD postgres "."
-amap_set SVC_TYPE minio docker; amap_set SVC_PORT minio 9000;
amap_set SVC_CWD minio "."
+amap_set SVC_TYPE rustfs docker; amap_set SVC_PORT rustfs 9000;
amap_set SVC_CWD rustfs "."
amap_set SVC_TYPE lakefs docker; amap_set SVC_PORT lakefs 8000;
amap_set SVC_CWD lakefs "."
amap_set SVC_TYPE lakekeeper docker; amap_set SVC_PORT lakekeeper 8181;
amap_set SVC_CWD lakekeeper "."
amap_set SVC_TYPE litellm docker; amap_set SVC_PORT litellm 4000;
amap_set SVC_CWD litellm "."
@@ -1003,8 +1003,8 @@ DOCKER_PROJECT="texera-local-dev"
DOCKER_COMPOSE_FILE="$SELF_ROOT/bin/single-node/docker-compose.yml"
DOCKER_OVERLAY_FILE="$SELF_ROOT/bin/local-dev/docker-compose.override.yml"
DOCKER_ENV_FILE="$SELF_ROOT/bin/single-node/.env"
-DOCKER_INFRA_SERVICES=(postgres minio minio-init lakefs lakekeeper-migrate
lakekeeper lakekeeper-init litellm jupyter)
-DOCKER_INFRA_LONGLIVED=(postgres minio lakefs lakekeeper litellm jupyter) #
exclude one-shot init jobs
+DOCKER_INFRA_SERVICES=(postgres rustfs rustfs-init lakefs lakekeeper-migrate
lakekeeper lakekeeper-init litellm jupyter)
+DOCKER_INFRA_LONGLIVED=(postgres rustfs lakefs lakekeeper litellm jupyter) #
exclude one-shot init jobs
# Build the array of -f flags: base single-node compose + local-dev overlay
# (the overlay publishes infra ports to the host, which the upstream compose
@@ -1361,7 +1361,7 @@ _install_hint() {
printf " Linux: see https://www.scala-sbt.org/download.html\n";
;;
docker)
- printf " ${BOLD}install Docker (needed for
postgres/minio/lakefs/lakekeeper/litellm):${RESET}\n"
+ printf " ${BOLD}install Docker (needed for
postgres/rustfs/lakefs/lakekeeper/litellm):${RESET}\n"
printf " macOS: download Docker Desktop from
https://docker.com/products/docker-desktop\n";
printf " Linux: apt install docker.io docker-compose-v2
${DIM}# dnf: moby-engine docker-compose${RESET}\n"
printf " then sudo usermod -aG docker \"\$USER\" and
log back in\n"
@@ -1651,7 +1651,7 @@ _refresh_docker_states_cache() {
fi
}
-# Per-service state for any of postgres/minio/lakefs/lakekeeper/litellm.
+# Per-service state for any of postgres/rustfs/lakefs/lakekeeper/litellm.
# Returns one of: running | starting | unhealthy | exited | failed | stopped
docker_svc_state() {
local svc="$1"
@@ -1684,7 +1684,7 @@ docker_svc_state() {
infra_up() {
# Resolve the host LAN IP now (lazy) — both the docker compose stack
# (lakekeeper-init reads STORAGE_S3_ENDPOINT) and the host JVMs about
- # to start need it pointing at a host-reachable MinIO.
+ # to start need it pointing at a host-reachable RustFS.
_require_host_lan_ip
if [[ "$(infra_state)" == external:* ]]; then
tui_err "infra: ports already taken by non-script containers"
@@ -1929,7 +1929,7 @@ infra_apply_sql_updates() {
#
# Used by cmd_auto, which only wants to touch what its scan said is
# dirty. cmd_up uses the heavier infra_up + infra_ensure_db_schema pair
-# instead so minio/lakefs/litellm warm up in parallel with the build.
+# instead so rustfs/lakefs/litellm warm up in parallel with the build.
ensure_postgres_for_build() {
if [[ "$(docker_svc_state postgres)" != "running" ]]; then
tui_step "postgres: starting (required for jOOQ codegen at build time)"
@@ -2179,7 +2179,7 @@ start_one() {
local type=""
type=$(amap_get SVC_TYPE "$svc")
# JVMs and docker services need STORAGE_S3_ENDPOINT exported before
- # launch (JVM clients dial MinIO; lakekeeper bakes the URL into the
+ # launch (JVM clients dial RustFS; lakekeeper bakes the URL into the
# warehouse storage profile). yarn/bun watch services don't.
if [[ "$type" == "jvm" || "$type" == "docker" ]]; then
_require_host_lan_ip
@@ -2886,7 +2886,7 @@ cmd_up() {
# generator returns Seq.empty and the downstream Scala compile fails
# on missing Tables/Keys/etc (the generated dir is not git-tracked).
# Bring infra up first so postgres is ready when the build fires.
- # As a bonus minio/lakefs/litellm warm up while sbt runs. (#6007)
+ # As a bonus rustfs/lakefs/litellm warm up while sbt runs. (#6007)
local svc=""
local has_docker_targets=false
for svc in "${SERVICES[@]}"; do
diff --git a/bin/local-dev/tui.py b/bin/local-dev/tui.py
index 8a5b7a5419..cdefdbd08a 100644
--- a/bin/local-dev/tui.py
+++ b/bin/local-dev/tui.py
@@ -279,7 +279,7 @@ def _jvm(name: str, port: int, project: Optional[str],
own_src: str) -> Service:
SERVICES: list[Service] = [
Service("postgres", "docker", 5432),
- Service("minio", "docker", 9000),
+ Service("rustfs", "docker", 9000),
Service("lakefs", "docker", 8000),
Service("lakekeeper", "docker", 8181),
Service("litellm", "docker", 4000),
diff --git a/bin/single-node/.env b/bin/single-node/.env
index 1f38de1002..69a9ceeecb 100644
--- a/bin/single-node/.env
+++ b/bin/single-node/.env
@@ -18,7 +18,7 @@
# Public host and ports exposed by the deployment
TEXERA_HOST=http://localhost
TEXERA_PORT=8080
-MINIO_PORT=9000
+RUSTFS_PORT=9000
JUPYTER_PORT=9100
# Log level for all Texera services (valid values: ERROR, WARN, INFO, DEBUG)
@@ -37,8 +37,8 @@ USER_SYS_ADMIN_PASSWORD=texera
POSTGRES_USER=texera
POSTGRES_PASSWORD=password
-# S3 (MinIO) credentials
-STORAGE_S3_AUTH_USERNAME=texera_minio
+# S3 (RustFS) credentials
+STORAGE_S3_AUTH_USERNAME=texera_rustfs
STORAGE_S3_AUTH_PASSWORD=password
# LakeFS server configuration
@@ -47,7 +47,7 @@ LAKEFS_INSTALLATION_ACCESS_KEY_ID=AKIAIOSFOLKFSSAMPLES
LAKEFS_INSTALLATION_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
LAKEFS_BLOCKSTORE_TYPE=s3
LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=true
-LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://texera-minio:9000
+LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://texera-rustfs:9000
LAKEFS_AUTH_ENCRYPT_SECRET_KEY=random_string_for_lakefs
LAKEFS_LOGGING_LEVEL=INFO
LAKEFS_STATS_ENABLED=1
@@ -61,7 +61,7 @@ LAKEKEEPER__PG_ENCRYPTION_KEY=texera_key
LAKEKEEPER_BASE_URI=http://texera-lakekeeper:8181
# Texera storage endpoints
-STORAGE_S3_ENDPOINT=http://texera-minio:9000
+STORAGE_S3_ENDPOINT=http://texera-rustfs:9000
STORAGE_S3_REGION=us-west-2
STORAGE_LAKEFS_ENDPOINT=http://texera-lakefs:8000/api/v1
STORAGE_JDBC_URL=jdbc:postgresql://texera-postgres:5432/texera_db?currentSchema=texera_db,public
diff --git a/bin/single-node/README.md b/bin/single-node/README.md
index 17ab8262ca..e3e8239192 100644
--- a/bin/single-node/README.md
+++ b/bin/single-node/README.md
@@ -159,12 +159,12 @@ All changes below are to the `.env` file in the
installation folder, unless othe
### Run Texera on other ports
By default, Texera uses:
- Port 8080 for its web service
-- Port 9000 for its MinIO storage service
+- Port 9000 for its RustFS storage service
- Port 9100 for the JupyterLab service used by the notebook migration tool
To change these ports, open the `.env` file and update the corresponding
variables:
- For the web service port (8080): change `TEXERA_PORT=8080` to your desired
port, e.g., `TEXERA_PORT=8081`.
-- For the MinIO port (9000): change `MINIO_PORT=9000` to your desired port,
e.g., `MINIO_PORT=9001`.
+- For the RustFS port (9000): change `RUSTFS_PORT=9000` to your desired port,
e.g., `RUSTFS_PORT=9001`.
- For the JupyterLab port (9100): change `JUPYTER_PORT=9100` to your desired
port, e.g., `JUPYTER_PORT=9101`.
### Change the locations of Texera data
diff --git a/bin/single-node/docker-compose.yml
b/bin/single-node/docker-compose.yml
index de6b45420b..f604db653a 100644
--- a/bin/single-node/docker-compose.yml
+++ b/bin/single-node/docker-compose.yml
@@ -18,33 +18,43 @@
name: texera-single-node
services:
# Part1: Specification of the storage services used by Texera
- # MinIO is an S3-compatible object storage used to store datasets and files.
- minio:
- image: minio/minio:RELEASE.2025-02-28T09-55-16Z
- container_name: texera-minio
+ # RustFS is an S3-compatible object storage used to store datasets and files.
+ rustfs:
+ image: rustfs/rustfs:1.0.0-rc.6
+ container_name: texera-rustfs
ports:
- - "${MINIO_PORT:-9000}:9000"
+ - "${RUSTFS_PORT:-9000}:9000"
env_file:
- .env
environment:
- - MINIO_ROOT_USER=${STORAGE_S3_AUTH_USERNAME}
- - MINIO_ROOT_PASSWORD=${STORAGE_S3_AUTH_PASSWORD}
+ - RUSTFS_ACCESS_KEY=${STORAGE_S3_AUTH_USERNAME}
+ - RUSTFS_SECRET_KEY=${STORAGE_S3_AUTH_PASSWORD}
+ # Must match STORAGE_S3_REGION: the region is part of the SigV4 scope,
and
+ # LakeFS sends its blockstore region on every request.
+ - RUSTFS_REGION=${STORAGE_S3_REGION}
+ - RUSTFS_CONSOLE_ENABLE=true
+ - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
+ # Empty log directory => log to stdout, so `docker compose logs rustfs`
+ # works and no second volume is needed. `warn` keeps that readable; the
+ # server is very chatty at info.
+ - RUSTFS_OBS_LOG_DIRECTORY=
+ - RUSTFS_OBS_LOGGER_LEVEL=warn
volumes:
- - minio_data:/data
- command: server --console-address ":9001" /data
+ - rustfs_data:/data
healthcheck:
- test: ["CMD", "curl", "-sf", "http://localhost:9000/minio/health/live";]
+ test: ["CMD", "curl", "-sf", "http://localhost:9000/health";]
interval: 5s
timeout: 3s
retries: 10
# This job creates the S3 bucket used by the Iceberg warehouse that the
- # Lakekeeper service manages.
- minio-init:
- image: minio/mc:RELEASE.2025-05-21T01-59-54Z
- container_name: texera-minio-init
+ # Lakekeeper service manages. `rc` is RustFS's S3 client, the counterpart of
+ # the `mc` client this used to run.
+ rustfs-init:
+ image: rustfs/rc:v0.1.35
+ container_name: texera-rustfs-init
depends_on:
- minio:
+ rustfs:
condition: service_healthy
env_file:
- .env
@@ -53,9 +63,10 @@ services:
command:
- |
set -e
- mc alias set local "$$STORAGE_S3_ENDPOINT"
"$$STORAGE_S3_AUTH_USERNAME" "$$STORAGE_S3_AUTH_PASSWORD"
- mc mb --ignore-existing
"local/$$STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET"
- echo "MinIO bucket '$$STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET' is
ready."
+ rc alias set local "$$STORAGE_S3_ENDPOINT"
"$$STORAGE_S3_AUTH_USERNAME" "$$STORAGE_S3_AUTH_PASSWORD" \
+ --region "$$STORAGE_S3_REGION" --bucket-lookup path
+ rc bucket create --ignore-existing
"local/$$STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET"
+ echo "RustFS bucket '$$STORAGE_ICEBERG_CATALOG_REST_S3_BUCKET' is
ready."
# PostgreSQL with PGroonga extension for full-text search.
# Used by lakeFS and Texera's metadata storage.
@@ -83,13 +94,13 @@ services:
depends_on:
postgres:
condition: service_healthy
- minio:
+ rustfs:
condition: service_started
env_file:
- .env
environment:
- # This port also need to be changed if the port of MinIO service is
changed
- -
LAKEFS_BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT=${TEXERA_HOST}:${MINIO_PORT:-9000}
+ # This port also need to be changed if the port of RustFS service is
changed
+ -
LAKEFS_BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT=${TEXERA_HOST}:${RUSTFS_PORT:-9000}
-
LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=${STORAGE_S3_AUTH_USERNAME}
-
LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=${STORAGE_S3_AUTH_PASSWORD}
entrypoint: ["/bin/sh", "-c"]
@@ -126,7 +137,7 @@ services:
depends_on:
postgres:
condition: service_healthy
- minio:
+ rustfs:
condition: service_started
lakekeeper-migrate:
condition: service_completed_successfully
@@ -142,14 +153,14 @@ services:
start_period: 10s
# One-shot init container that creates the Lakekeeper default project and
- # the Iceberg warehouse pointing at the MinIO bucket prepared by minio-init.
+ # the Iceberg warehouse pointing at the RustFS bucket prepared by
rustfs-init.
lakekeeper-init:
image: alpine:3.19
container_name: texera-lakekeeper-init
depends_on:
lakekeeper:
condition: service_healthy
- minio-init:
+ rustfs-init:
condition: service_completed_successfully
env_file:
- .env
@@ -222,7 +233,7 @@ services:
# The warehouse persists its S3 endpoint in Lakekeeper's own DB. The
# local-dev launcher sets STORAGE_S3_ENDPOINT to the host LAN IP so
# both this container and the host JVMs (Iceberg remote-signing)
reach
- # the same MinIO — but that IP changes across networks / DHCP leases.
+ # the same RustFS — but that IP changes across networks / DHCP
leases.
# This init is idempotent and would otherwise just skip, leaving a
# stale endpoint that breaks workflow execution with an opaque
Iceberg
# RESTException. Refresh the stored endpoint on every run (#6195).
@@ -314,7 +325,7 @@ services:
container_name: file-service
restart: unless-stopped
depends_on:
- minio:
+ rustfs:
condition: service_started
lakefs:
condition: service_healthy
@@ -584,6 +595,6 @@ networks:
# persistent volumes
volumes:
- minio_data:
+ rustfs_data:
postgres_data:
workflow_result_data:
\ No newline at end of file
diff --git a/build.sbt b/build.sbt
index 02085096c5..a5b1bd9bf3 100644
--- a/build.sbt
+++ b/build.sbt
@@ -213,6 +213,7 @@ lazy val FileService = (project in file("file-service"))
.dependsOn(WorkflowCore, Auth, Config, Resource, Util)
.configs(Test)
.dependsOn(DAO % "test->test") // test scope dependency
+ .dependsOn(WorkflowCore % "test->test") // reuse RustFSContainer in
MockLakeFS
.settings(
dependencyOverrides ++= Seq(
// override it as io.dropwizard 4 require 2.16.1 or higher
@@ -220,7 +221,7 @@ lazy val FileService = (project in file("file-service"))
"com.fasterxml.jackson.core" % "jackson-databind" % jacksonVersion,
"org.glassfish.jersey.core" % "jersey-common" % "3.0.12"
) ++ nettyDependencyOverrides,
- // Each testcontainers-based suite starts its own LakeFS/MinIO/Postgres
stack
+ // Each testcontainers-based suite starts its own LakeFS/RustFS/Postgres
stack
// and mutates JVM-wide singletons (StorageConfig endpoints, LakeFS
client),
// so every suite gets its own forked JVM; sbt runs forked groups one at a
// time by default (Tags.ForkedTestGroup limit), keeping the stacks serial.
diff --git a/common/config/src/main/resources/storage.conf
b/common/config/src/main/resources/storage.conf
index 9af2924901..65f905566b 100644
--- a/common/config/src/main/resources/storage.conf
+++ b/common/config/src/main/resources/storage.conf
@@ -125,7 +125,7 @@ storage {
}
auth {
- username = "texera_minio"
+ username = "texera_rustfs"
username = ${?STORAGE_S3_AUTH_USERNAME}
password = "password"
diff --git a/common/workflow-core/build.sbt b/common/workflow-core/build.sbt
index bfc1467041..c6f0a82ce8 100644
--- a/common/workflow-core/build.sbt
+++ b/common/workflow-core/build.sbt
@@ -35,9 +35,9 @@ ThisBuild / conflictManager := ConflictManager.latestRevision
Global / concurrentRestrictions += Tags.limit(Tags.Test, 1)
// Suites tagged @org.apache.texera.common.tags.NonParallelTest must not run
concurrently with one
-// another. They share a JVM-wide singleton backed by an external resource —
the MinIO-backed
+// another. They share a JVM-wide singleton backed by an external resource —
the RustFS-backed
// suites mixing S3StorageTestBase share one S3StorageClient.s3Client and one
-// StorageConfig.s3Endpoint pointed at a single MinIO container — so
ScalaTest's parallel suite
+// StorageConfig.s3Endpoint pointed at a single RustFS container — so
ScalaTest's parallel suite
// distributor otherwise runs them together, they contend, and intermittently
time out (flaky; see
// issue #7049). The Global Tags.limit(Tags.Test, 1) above only bounds sbt
task concurrency, not
// ScalaTest's in-JVM distributor.
@@ -125,8 +125,9 @@ libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "3.2.15" % Test, //
ScalaTest
"junit" % "junit" % "4.13.2" % Test, // JUnit
"com.novocode" % "junit-interface" % "0.11" % Test, // SBT
interface for JUnit
+ // RustFS has no dedicated testcontainers-scala module; RustFSContainer
builds it on the
+ // GenericContainer that testcontainers-scala-core (pulled in by -scalatest)
provides.
"com.dimafeng" %% "testcontainers-scala-scalatest" % testcontainersVersion %
Test, // Testcontainers ScalaTest integration
- "com.dimafeng" %% "testcontainers-scala-minio" % testcontainersVersion %
Test, // MinIO Testcontainer Scala integration
"com.dimafeng" %% "testcontainers-scala-postgresql" % testcontainersVersion
% Test // Postgres Testcontainer (LakeFS metadata store)
)
diff --git
a/common/workflow-core/src/main/scala/org/apache/texera/service/util/S3StorageClient.scala
b/common/workflow-core/src/main/scala/org/apache/texera/service/util/S3StorageClient.scala
index 8be8c7c701..38b839d6aa 100644
---
a/common/workflow-core/src/main/scala/org/apache/texera/service/util/S3StorageClient.scala
+++
b/common/workflow-core/src/main/scala/org/apache/texera/service/util/S3StorageClient.scala
@@ -30,7 +30,7 @@ import java.io.InputStream
import scala.jdk.CollectionConverters._
/**
- * S3Storage provides an abstraction for S3-compatible storage (e.g., MinIO).
+ * S3Storage provides an abstraction for S3-compatible storage (e.g., RustFS).
* - Uses credentials and endpoint from StorageConfig.
* - Supports object upload, download, listing, and deletion.
*/
@@ -44,14 +44,14 @@ object S3StorageClient {
// Cap how many failed keys are listed in the error message.
private[util] val MAX_LISTED_DELETE_ERRORS = 10
- // Initialize MinIO-compatible S3 Client
+ // Initialize the S3 client for the configured S3-compatible endpoint
private lazy val s3Client: S3Client = {
val credentials = AwsBasicCredentials.create(StorageConfig.s3Username,
StorageConfig.s3Password)
S3Client
.builder()
.credentialsProvider(StaticCredentialsProvider.create(credentials))
.region(Region.of(StorageConfig.s3Region))
- .endpointOverride(java.net.URI.create(StorageConfig.s3Endpoint)) //
MinIO URL
+ .endpointOverride(java.net.URI.create(StorageConfig.s3Endpoint)) //
object-store URL
.serviceConfiguration(
S3Configuration.builder().pathStyleAccessEnabled(true).build()
)
@@ -106,7 +106,7 @@ object S3StorageClient {
* A trailing `/` is added when missing so the prefix matches on a path
boundary (`a/b` deletes
* `a/b/file` but not `a/bc/file`). An empty prefix would match the whole
bucket and is rejected.
*
- * @param bucketName Target S3/MinIO bucket.
+ * @param bucketName Target S3 bucket.
* @param directoryPrefix Non-empty key prefix to delete.
*/
def deleteDirectory(bucketName: String, directoryPrefix: String): Unit = {
diff --git
a/common/workflow-core/src/test/java/org/apache/texera/common/tags/NonParallelTest.java
b/common/workflow-core/src/test/java/org/apache/texera/common/tags/NonParallelTest.java
index 249fc4dce5..5ab8de24b3 100644
---
a/common/workflow-core/src/test/java/org/apache/texera/common/tags/NonParallelTest.java
+++
b/common/workflow-core/src/test/java/org/apache/texera/common/tags/NonParallelTest.java
@@ -34,7 +34,7 @@ import org.scalatest.TagAnnotation;
* runs in parallel in a single shared group.
*
* <p>Use this for suites that share a JVM-wide singleton backed by an
external resource and would
- * otherwise contend when the ScalaTest distributor runs them in parallel —
e.g. the MinIO-backed
+ * otherwise contend when the ScalaTest distributor runs them in parallel —
e.g. the RustFS-backed
* suites mixing {@code S3StorageTestBase}, which share one {@code
S3StorageClient.s3Client} and
* {@code StorageConfig} endpoint (see issue #7049).
*
diff --git
a/common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClientMtimeSpec.scala
b/common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClientMtimeSpec.scala
index c08ad1d6c2..88076d807c 100644
---
a/common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClientMtimeSpec.scala
+++
b/common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/util/LakeFSStorageClientMtimeSpec.scala
@@ -22,13 +22,13 @@ package org.apache.texera.amber.core.storage.util
import com.dimafeng.testcontainers.{
ForAllTestContainer,
GenericContainer,
- MinIOContainer,
MultipleContainers,
PostgreSQLContainer
}
import io.lakefs.clients.sdk.ApiException
import org.apache.texera.common.config.StorageConfig
import org.apache.texera.common.tags.NonParallelTest
+import org.apache.texera.service.util.RustFSContainer
import org.scalatest.flatspec.AnyFlatSpec
import org.scalatest.matchers.should.Matchers
import org.testcontainers.containers.Network
@@ -46,7 +46,7 @@ import java.nio.charset.StandardCharsets
* Spec for [[LakeFSStorageClient.getStagedObjectMtime]].
*
* The method is a thin LakeFS-SDK passthrough (statObject -> mtime), so it
can only be
- * exercised against a real LakeFS. This spins up the same Postgres + MinIO +
LakeFS stack
+ * exercised against a real LakeFS. This spins up the same Postgres + RustFS
+ LakeFS stack
* the file-service tests use, with Postgres backing the LakeFS metadata
store.
*
* Tagged [[NonParallelTest]] so `common/workflow-core/build.sbt` gives this
suite its own forked
@@ -63,12 +63,12 @@ class LakeFSStorageClientMtimeSpec
with ForAllTestContainer
with org.scalatest.BeforeAndAfterAll {
- // Shared network so LakeFS can reach Postgres and MinIO by their in-network
aliases while
- // the test reaches LakeFS/MinIO via mapped host ports.
+ // Shared network so LakeFS can reach Postgres and RustFS by their
in-network aliases while
+ // the test reaches LakeFS/RustFS via mapped host ports.
private val network: Network = Network.newNetwork()
- private val minioUser = "texera_minio"
- private val minioPassword = "password"
+ private val s3User = RustFSContainer.DefaultUser
+ private val s3Password = RustFSContainer.DefaultPassword
// Postgres metadata store for LakeFS. Using a real DB (rather than LakeFS's
local/quickstart
// KV) keeps setup explicit and deterministic: local mode auto-initializes
on boot, which then
@@ -87,12 +87,12 @@ class LakeFSStorageClientMtimeSpec
s"postgresql://${postgres.username}:${postgres.password}" +
s"@${postgres.container.getNetworkAliases.get(0)}:5432/${postgres.databaseName}?sslmode=disable"
- private val minio: MinIOContainer = MinIOContainer(
- dockerImageName =
DockerImageName.parse("minio/minio:RELEASE.2025-02-28T09-55-16Z"),
- userName = minioUser,
- password = minioPassword
+ private val objectStore: GenericContainer = RustFSContainer(
+ userName = s3User,
+ password = s3Password,
+ region = Region.US_WEST_2.id()
)
- minio.container.withNetwork(network)
+ objectStore.container.withNetwork(network)
private val lakefs: GenericContainer = GenericContainer(
dockerImage = "treeverse/lakefs:1.51",
@@ -102,9 +102,10 @@ class LakeFSStorageClientMtimeSpec
"LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING" -> lakefsDatabaseURL,
"LAKEFS_BLOCKSTORE_TYPE" -> "s3",
"LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE" -> "true",
- "LAKEFS_BLOCKSTORE_S3_ENDPOINT" ->
s"http://${minio.container.getNetworkAliases.get(0)}:9000",
- "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID" -> minioUser,
- "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY" -> minioPassword,
+ "LAKEFS_BLOCKSTORE_S3_ENDPOINT" ->
s"http://${objectStore.container.getNetworkAliases
+ .get(0)}:${RustFSContainer.Port}",
+ "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID" -> s3User,
+ "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY" -> s3Password,
"LAKEFS_AUTH_ENCRYPT_SECRET_KEY" -> "random_string_for_lakefs",
"LAKEFS_INSTALLATION_USER_NAME" -> "texera-admin",
"LAKEFS_INSTALLATION_ACCESS_KEY_ID" -> StorageConfig.lakefsUsername,
@@ -113,9 +114,11 @@ class LakeFSStorageClientMtimeSpec
)
lakefs.container.withNetwork(network)
- override val container: MultipleContainers = MultipleContainers(postgres,
minio, lakefs)
+ override val container: MultipleContainers =
+ MultipleContainers(postgres, objectStore, lakefs)
- private def minioEndpoint: String =
s"http://${minio.host}:${minio.mappedPort(9000)}"
+ private def s3Endpoint: String =
+
s"http://${objectStore.host}:${objectStore.mappedPort(RustFSContainer.Port)}"
private def lakefsApiBasePath: String =
s"http://${lakefs.host}:${lakefs.mappedPort(8000)}/api/v1"
override def afterStart(): Unit = {
@@ -138,7 +141,7 @@ class LakeFSStorageClientMtimeSpec
// Point the JVM-wide singletons at the test containers BEFORE any
LakeFSStorageClient call,
// since its api client is a lazy val that captures the endpoint on first
use.
- StorageConfig.s3Endpoint = minioEndpoint
+ StorageConfig.s3Endpoint = s3Endpoint
StorageConfig.lakefsEndpoint = lakefsApiBasePath
// LakeFS needs its blockstore bucket to exist before any repo can be
created.
@@ -148,10 +151,10 @@ class LakeFSStorageClientMtimeSpec
private def createLakefsBucket(): Unit = {
val s3 = S3Client
.builder()
- .endpointOverride(URI.create(minioEndpoint))
- .region(Region.US_WEST_2) // required by the builder; irrelevant for
MinIO
+ .endpointOverride(URI.create(s3Endpoint))
+ .region(Region.US_WEST_2) // must match RUSTFS_REGION: it is part of the
SigV4 scope
.credentialsProvider(
- StaticCredentialsProvider.create(AwsBasicCredentials.create(minioUser,
minioPassword))
+ StaticCredentialsProvider.create(AwsBasicCredentials.create(s3User,
s3Password))
)
.serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).build())
.build()
diff --git
a/common/workflow-core/src/test/scala/org/apache/texera/service/util/RustFSContainer.scala
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/RustFSContainer.scala
new file mode 100644
index 0000000000..fd07258f73
--- /dev/null
+++
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/RustFSContainer.scala
@@ -0,0 +1,80 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.texera.service.util
+
+import com.dimafeng.testcontainers.GenericContainer
+import org.testcontainers.containers.wait.strategy.Wait
+
+import java.time.Duration
+
+/**
+ * Factory for the RustFS container used by the S3-backed test suites.
+ *
+ * testcontainers-scala ships a `MinIOContainer` module but no RustFS one, so
this builds the
+ * equivalent on top of `GenericContainer`. The differences from
`MinIOContainer` that matter:
+ *
+ * - credentials come from `RUSTFS_ACCESS_KEY` / `RUSTFS_SECRET_KEY` rather
than
+ * `MINIO_ROOT_USER` / `MINIO_ROOT_PASSWORD`;
+ * - readiness is `GET /health` on the S3 port, not `GET
/minio/health/live`;
+ * - `RUSTFS_REGION` must match the region the AWS SDK signs with, because
the region is part
+ * of the SigV4 credential scope.
+ *
+ * The console (9001) is left disabled: no test drives it, and starting it
only widens the
+ * surface the readiness check has to wait for.
+ */
+object RustFSContainer {
+
+ /** Image pinned in lockstep with `bin/single-node/docker-compose.yml` and
`bin/k8s/values.yaml`. */
+ val ImageName: String = "rustfs/rustfs:1.0.0-rc.6"
+
+ /** S3 API port inside the container. */
+ val Port: Int = 9000
+
+ val DefaultUser: String = "texera_rustfs"
+ val DefaultPassword: String = "password"
+ val DefaultRegion: String = "us-west-2"
+
+ def apply(
+ userName: String = DefaultUser,
+ password: String = DefaultPassword,
+ region: String = DefaultRegion
+ ): GenericContainer = {
+ val container = GenericContainer(
+ dockerImage = ImageName,
+ exposedPorts = Seq(Port),
+ env = Map(
+ "RUSTFS_ACCESS_KEY" -> userName,
+ "RUSTFS_SECRET_KEY" -> password,
+ "RUSTFS_REGION" -> region,
+ "RUSTFS_VOLUMES" -> "/data",
+ // Log to stdout so a failed start shows up in the testcontainers log
consumer
+ // instead of a file inside the container. `warn` keeps that readable.
+ "RUSTFS_OBS_LOG_DIRECTORY" -> "",
+ "RUSTFS_OBS_LOGGER_LEVEL" -> "warn"
+ ),
+ waitStrategy = Wait
+ .forHttp("/health")
+ .forPort(Port)
+ .forStatusCode(200)
+ .withStartupTimeout(Duration.ofMinutes(2))
+ )
+ container
+ }
+}
diff --git
a/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageClientSpec.scala
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageClientSpec.scala
index 78765381ff..1e910a46b6 100644
---
a/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageClientSpec.scala
+++
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageClientSpec.scala
@@ -63,7 +63,7 @@ class S3StorageClientSpec
}
/**
- * A second S3 client pointed at the same MinIO container. S3StorageClient
exposes only
+ * A second S3 client pointed at the same RustFS container. S3StorageClient
exposes only
* `uploadPartWithRequest` from the multipart API, so the surrounding
create/complete/list
* calls are issued directly instead of being mocked away.
*/
@@ -438,7 +438,7 @@ class S3StorageClientSpec
val objectCount = 1001
// Upload with bounded concurrency to keep the test reasonably fast
without flooding the
- // shared MinIO container (a 16-way burst was a contributor to the
flakiness in issue #7049).
+ // shared RustFS container (a 16-way burst was a contributor to the
flakiness in issue #7049).
val pool = Executors.newFixedThreadPool(4)
implicit val ec: ExecutionContext = ExecutionContext.fromExecutor(pool)
try {
diff --git
a/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageTestBase.scala
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageTestBase.scala
index a6e5f9a5c8..38901ba0ca 100644
---
a/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageTestBase.scala
+++
b/common/workflow-core/src/test/scala/org/apache/texera/service/util/S3StorageTestBase.scala
@@ -19,14 +19,13 @@
package org.apache.texera.service.util
-import com.dimafeng.testcontainers.MinIOContainer
+import com.dimafeng.testcontainers.GenericContainer
import org.apache.texera.common.config.StorageConfig
import org.scalatest.{BeforeAndAfterAll, Suite}
-import org.testcontainers.utility.DockerImageName
/**
- * Base trait for tests requiring S3 storage (MinIO).
- * Provides access to a single shared MinIO container across all test suites.
+ * Base trait for tests requiring S3 storage (RustFS).
+ * Provides access to a single shared RustFS container across all test suites.
*
* Usage: Mix this trait into any test suite that needs S3 storage.
*/
@@ -40,21 +39,17 @@ trait S3StorageTestBase extends BeforeAndAfterAll { this:
Suite =>
}
object S3StorageTestBase {
- private lazy val container: MinIOContainer = {
- val c = MinIOContainer(
- dockerImageName =
DockerImageName.parse("minio/minio:RELEASE.2025-02-28T09-55-16Z"),
- userName = "texera_minio",
- password = "password"
- )
+ private lazy val container: GenericContainer = {
+ val c = RustFSContainer(region = StorageConfig.s3Region)
c.start()
- val endpoint = s"http://${c.host}:${c.mappedPort(9000)}"
+ val endpoint = s"http://${c.host}:${c.mappedPort(RustFSContainer.Port)}"
StorageConfig.s3Endpoint = endpoint
- println(s"[S3Storage] Started shared MinIO at $endpoint")
+ println(s"[S3Storage] Started shared RustFS at $endpoint")
sys.addShutdownHook {
- println("[S3Storage] Stopping shared MinIO...")
+ println("[S3Storage] Stopping shared RustFS...")
c.stop()
}
diff --git a/docs/contribution-guidelines/guide-for-developers.md
b/docs/contribution-guidelines/guide-for-developers.md
index 077746390d..5a58c22a95 100644
--- a/docs/contribution-guidelines/guide-for-developers.md
+++ b/docs/contribution-guidelines/guide-for-developers.md
@@ -147,9 +147,9 @@ Execute `sql/iceberg_postgres_catalog.sql` to create the
database for storing I
psql -U postgres -f "sql/iceberg_postgres_catalog.sql"
```
-### Setup the LakeFS+Minio locally
+### Setup the LakeFS+RustFS locally
-Texera requires [LakeFS](https://lakefs.io/) and
S3([Minio](https://min.io/docs/minio/kubernetes/upstream/index.html) is one of
the implementations) as the dataset storage. Setting up these two storage
services locally are required to make Texera's dataset feature functioning.
+Texera requires [LakeFS](https://lakefs.io/) and
S3([RustFS](https://docs.rustfs.com/) is one of the implementations) as the
dataset storage. Setting up these two storage services locally are required to
make Texera's dataset feature functioning.
Install [Docker
Desktop](https://docs.docker.com/desktop/setup/install/mac-install/) which
contains both docker engine and docker compose. Make sure you launch the Docker
after installing it.
@@ -160,7 +160,7 @@ cd file-service/src/main/resources
Edit `docker-compose.yml` by: search for `volumes` in the file and follow the
instructions in the comment. This step is required otherwise your data will be
lost if containers are deleted
-Execute the following command to start LakeFS and Minio:
+Execute the following command to start LakeFS and RustFS:
```
docker compose up
```
diff --git a/docs/getting-started/installing-using-docker.md
b/docs/getting-started/installing-using-docker.md
index b77e446c0e..335667083c 100644
--- a/docs/getting-started/installing-using-docker.md
+++ b/docs/getting-started/installing-using-docker.md
@@ -162,11 +162,11 @@ All changes below are to the `.env` file in the
installation folder, unless othe
### Run Texera on other ports
By default, Texera uses:
- Port 8080 for its web service
-- Port 9000 for its MinIO storage service
+- Port 9000 for its RustFS storage service
To change these ports, open the `.env` file and update the corresponding
variables:
- For the web service port (8080): change `TEXERA_PORT=8080` to your desired
port, e.g., `TEXERA_PORT=8081`.
-- For the MinIO port (9000): change `MINIO_PORT=9000` to your desired port,
e.g., `MINIO_PORT=9001`.
+- For the RustFS port (9000): change `RUSTFS_PORT=9000` to your desired port,
e.g., `RUSTFS_PORT=9001`.
### Change the locations of Texera data
By default, Docker manages Texera's data locations. To change them to your own
locations:
diff --git a/docs/getting-started/run-on-kubernetes.md
b/docs/getting-started/run-on-kubernetes.md
index 6503b8dc91..d51c3e3ce0 100644
--- a/docs/getting-started/run-on-kubernetes.md
+++ b/docs/getting-started/run-on-kubernetes.md
@@ -110,12 +110,12 @@ Once the deployments are running, you can access the
Texera web interface.
### File Upload Error
-If you see an error when trying to upload a file to a dataset, you may need to
forward the port for MinIO (our file storage service).
+If you see an error when trying to upload a file to a dataset, you may need to
forward the port for RustFS (our file storage service).
Run the following command in a new terminal:
```bash
-kubectl port-forward -n texera-dev service/texera-minio 31000:9000
+kubectl port-forward -n texera-dev service/texera-rustfs-svc 31000:9000
```
This maps the service's port `9000` to your local port `31000`.
diff --git a/docs/overview.md b/docs/overview.md
index 2b232db723..08e1adde76 100644
--- a/docs/overview.md
+++ b/docs/overview.md
@@ -37,7 +37,7 @@ At its core, Texera acts as a bridge between a highly
accessible frontend and a
1. **Web-Based Interface (Frontend):** A rich GUI running directly in your
browser. It allows users to construct data processing pipelines by dragging and
dropping blocks on a canvas. No installation is required on client machines.
2. **Distributed Engine (Backend):** When a workflow is submitted, the Texera
engine compiles the graphical representation into an optimized, distributed
execution plan. It then spins up computing units to process massive datasets in
parallel.
-3. **Storage Integration:** Texera integrates smoothly with modern data lake
and storage technologies (like LakeFS and MinIO) to persistently log runs and
save datasets securely.
+3. **Storage Integration:** Texera integrates smoothly with modern data lake
and storage technologies (like LakeFS and RustFS) to persistently log runs and
save datasets securely.
---
diff --git a/file-service/build.sbt b/file-service/build.sbt
index defbb99c1f..f060aff17f 100644
--- a/file-service/build.sbt
+++ b/file-service/build.sbt
@@ -73,9 +73,10 @@ libraryDependencies ++= Seq(
"org.mockito" % "mockito-core" % mockitoVersion % Test, //
Mockito for mocking
"org.assertj" % "assertj-core" % assertjVersion % Test, //
AssertJ for assertions
"com.novocode" % "junit-interface" % "0.11" % Test, // SBT
interface for JUnit
+ // RustFS has no dedicated testcontainers-scala module; MockLakeFS uses the
shared
+ // RustFSContainer from WorkflowCore's test sources (see the test->test dep
in build.sbt).
"com.dimafeng" %% "testcontainers-scala-scalatest" % testcontainersVersion %
Test, // Testcontainers ScalaTest integration
"com.dimafeng" %% "testcontainers-scala-postgresql" % testcontainersVersion
% Test, // PostgreSQL Testcontainer Scala integration
- "com.dimafeng" %% "testcontainers-scala-minio" % testcontainersVersion %
Test, // MinIO Testcontainer Scala integration
)
/////////////////////////////////////////////////////////////////////////////
diff --git a/file-service/src/main/resources/docker-compose.yml
b/file-service/src/main/resources/docker-compose.yml
index f4d2a07256..95088150b1 100644
--- a/file-service/src/main/resources/docker-compose.yml
+++ b/file-service/src/main/resources/docker-compose.yml
@@ -17,20 +17,23 @@
name: texera-lakefs
services:
- minio:
- image: minio/minio:RELEASE.2025-02-28T09-55-16Z
- container_name: texera-lakefs-minio
+ rustfs:
+ image: rustfs/rustfs:1.0.0-rc.6
+ container_name: texera-lakefs-rustfs
restart: unless-stopped
ports:
- "9000:9000"
- "9001:9001"
environment:
- - MINIO_ROOT_USER=texera_minio
- - MINIO_ROOT_PASSWORD=password
- command: server --console-address ":9001" /data
+ - RUSTFS_ACCESS_KEY=texera_rustfs
+ - RUSTFS_SECRET_KEY=password
+ # Must match the region the AWS SDK signs with: it is part of the SigV4
scope.
+ - RUSTFS_REGION=us-west-2
+ - RUSTFS_CONSOLE_ENABLE=true
+ - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
# The lines below are recommended to mount a host path in order to persist
your data even if the container is removed.
volumes:
- - minio_data:/data
+ - rustfs_data:/data
postgres:
image: postgres:15
@@ -56,16 +59,16 @@ services:
depends_on:
postgres:
condition: service_healthy
- minio:
+ rustfs:
condition: service_started
ports:
- "8000:8000"
environment:
- LAKEFS_BLOCKSTORE_TYPE=s3
- LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=true
- - LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://minio:9000
+ - LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://rustfs:9000
- LAKEFS_BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT=http://localhost:9000
- - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=texera_minio
+ - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=texera_rustfs
- LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=password
- LAKEFS_AUTH_ENCRYPT_SECRET_KEY=random_string_for_lakefs
- LAKEFS_LOGGING_LEVEL=INFO
@@ -94,5 +97,5 @@ networks:
# Named Docker volumes — uncomment the following lines if you mounted host
paths above.
volumes:
- minio_data:
+ rustfs_data:
postgres_data:
diff --git a/file-service/src/main/resources/minio-config.yml
b/file-service/src/main/resources/rustfs-config.yml
similarity index 69%
rename from file-service/src/main/resources/minio-config.yml
rename to file-service/src/main/resources/rustfs-config.yml
index 0622b92bdc..f29fed9e27 100644
--- a/file-service/src/main/resources/minio-config.yml
+++ b/file-service/src/main/resources/rustfs-config.yml
@@ -18,15 +18,20 @@
version: '3.8'
services:
- minio:
- image: minio/minio:latest
- container_name: minio
+ rustfs:
+ image: rustfs/rustfs:1.0.0-rc.6
+ container_name: rustfs
ports:
- - "9500:9000" # MinIO API
- - "9501:9001" # MinIO Console UI
+ - "9500:9000" # S3 API
+ - "9501:9001" # Console UI
environment:
- - MINIO_ROOT_USER=texera_minio
- - MINIO_ROOT_PASSWORD=password
+ - RUSTFS_ACCESS_KEY=texera_rustfs
+ - RUSTFS_SECRET_KEY=password
+ - RUSTFS_REGION=us-west-2
+ - RUSTFS_CONSOLE_ENABLE=true
+ - RUSTFS_CONSOLE_ADDRESS=0.0.0.0:9001
volumes:
- -
/Users/baijiadong/Desktop/chenlab/texera/core/file-service/src/main/user-resources/minio:/data
- command: server --console-address ":9001" /data
\ No newline at end of file
+ - rustfs_data:/data
+
+volumes:
+ rustfs_data:
\ No newline at end of file
diff --git
a/file-service/src/main/scala/org/apache/texera/service/util/StagedFileCleanupJob.scala
b/file-service/src/main/scala/org/apache/texera/service/util/StagedFileCleanupJob.scala
index 9e2ed257ba..b509014de4 100644
---
a/file-service/src/main/scala/org/apache/texera/service/util/StagedFileCleanupJob.scala
+++
b/file-service/src/main/scala/org/apache/texera/service/util/StagedFileCleanupJob.scala
@@ -44,6 +44,29 @@ case class CleanupReport(sessionsDeleted: Int, objectsReset:
Int, errors: Int)
object StagedFileCleanupJob {
private[util] val DefaultSessionCleanupBatchSize = 500
+
+ /**
+ * Whether a failed LakeFS multipart abort means "this upload is already
gone", which the
+ * cleanup job treats as success rather than as an error to retry.
+ *
+ * Two shapes have to be recognised because LakeFS does not normalise its
object store's
+ * answer:
+ *
+ * - `404` — LakeFS itself has no record of the upload.
+ * - `500` whose body carries the object store's `NoSuchUpload` — LakeFS
forwarded the
+ * abort and the store reported the upload gone. LakeFS wraps that
backend status in a
+ * 500 rather than passing the 404 through.
+ *
+ * The second case is what a spec-conforming store returns:
`AbortMultipartUpload` on an
+ * unknown upload id is a `NoSuchUpload` error in the S3 API. (MinIO, the
store Texera used
+ * before RustFS, answered such an abort with success instead, so this path
never ran.)
+ * Matching on the `NoSuchUpload` code rather than on the bare 500 keeps
every other server
+ * error an error, so a store that is merely unreachable still rolls the
transaction back
+ * and is retried next round.
+ */
+ private[util] def isAlreadyAborted(e: ApiException): Boolean =
+ e.getCode == 404 ||
+ (e.getCode == 500 &&
Option(e.getResponseBody).exists(_.contains("NoSuchUpload")))
}
/**
@@ -149,10 +172,11 @@ class StagedFileCleanupJob(
try {
// Delete the row and abort the multipart in one transaction, deleting
FIRST. LakeFS is
// external and cannot truly enroll in a DB transaction, but the abort
is idempotent
- // (re-aborting an already-aborted upload returns 404, treated as
success below), so the
- // only risk is the abort failing AFTER the delete is staged. By
staging the delete first
- // and letting a non-404 abort failure roll the whole transaction
back, the session row
- // survives and the next round retries — never leaving an orphaned
multipart behind.
+ // (re-aborting an already-aborted upload is treated as success below,
see
+ // `isAlreadyAborted`), so the only risk is the abort failing AFTER
the delete is staged.
+ // By staging the delete first and letting a genuine abort failure
roll the whole
+ // transaction back, the session row survives and the next round
retries — never leaving
+ // an orphaned multipart behind.
SqlServer.withTransaction(ctx) { txn =>
txn
.deleteFrom(DATASET_UPLOAD_SESSION)
@@ -169,7 +193,7 @@ class StagedFileCleanupJob(
)
} catch {
// Already aborted (or never materialized): safe to delete the
session row.
- case e: ApiException if e.getCode == 404 =>
+ case e: ApiException if
StagedFileCleanupJob.isAlreadyAborted(e) =>
logger.debug(
s"Multipart upload ${session.getUploadId} not found in
LakeFS; " +
"treating as already aborted"
diff --git
a/file-service/src/test/scala/org/apache/texera/service/MockLakeFS.scala
b/file-service/src/test/scala/org/apache/texera/service/MockLakeFS.scala
index 3f5ae772ec..a1c1b1a135 100644
--- a/file-service/src/test/scala/org/apache/texera/service/MockLakeFS.scala
+++ b/file-service/src/test/scala/org/apache/texera/service/MockLakeFS.scala
@@ -22,7 +22,7 @@ package org.apache.texera.service
import com.dimafeng.testcontainers._
import io.lakefs.clients.sdk.{ApiClient, RepositoriesApi}
import org.apache.texera.common.config.StorageConfig
-import org.apache.texera.service.util.S3StorageClient
+import org.apache.texera.service.util.{RustFSContainer, S3StorageClient}
import org.scalatest.{BeforeAndAfterAll, Suite}
import org.testcontainers.containers.Network
import org.testcontainers.utility.DockerImageName
@@ -34,7 +34,7 @@ import software.amazon.awssdk.services.s3.S3Configuration
import java.net.URI
/**
- * Trait to spin up a LakeFS + MinIO + Postgres stack using Testcontainers,
+ * Trait to spin up a LakeFS + RustFS + Postgres stack using Testcontainers,
* similar to how MockTexeraDB uses EmbeddedPostgres.
*/
trait MockLakeFS extends ForAllTestContainer with BeforeAndAfterAll { self:
Suite =>
@@ -53,29 +53,27 @@ trait MockLakeFS extends ForAllTestContainer with
BeforeAndAfterAll { self: Suit
postgres.container.withNetwork(network)
// LakeFS bakes the pre-signed endpoint into its env before containers start,
- // so MinIO cannot use a dynamically mapped host port: presigned URLs must be
- // reachable from the host at an address known ahead of time. Reserve a free
- // host port and pin MinIO's 9000 to it.
- val minioHostPort: Int = {
+ // so the object store cannot use a dynamically mapped host port: presigned
URLs
+ // must be reachable from the host at an address known ahead of time.
Reserve a
+ // free host port and pin the S3 port to it.
+ val objectStoreHostPort: Int = {
val socket = new java.net.ServerSocket(0)
try socket.getLocalPort
finally socket.close()
}
- // MinIO for object storage
- val minio = MinIOContainer(
- dockerImageName =
DockerImageName.parse("minio/minio:RELEASE.2025-02-28T09-55-16Z"),
- userName = "texera_minio",
- password = "password"
- )
- minio.container.withNetwork(network)
- minio.container.withCreateContainerCmdModifier { cmd =>
+ // RustFS for object storage
+ val objectStore: GenericContainer = RustFSContainer()
+ objectStore.container.withNetwork(network)
+ objectStore.container.withCreateContainerCmdModifier { cmd =>
import com.github.dockerjava.api.model.{ExposedPort, PortBinding, Ports}
- // setting explicit bindings replaces them all, so 9001 (console) must keep
- // a dynamic binding or the container readiness check never passes
+ // RustFSContainer exposes only the S3 port, so pinning it is the whole
+ // binding set; the console is not started.
cmd.getHostConfig.withPortBindings(
- new PortBinding(Ports.Binding.bindPort(minioHostPort),
ExposedPort.tcp(9000)),
- new PortBinding(Ports.Binding.empty(), ExposedPort.tcp(9001))
+ new PortBinding(
+ Ports.Binding.bindPort(objectStoreHostPort),
+ ExposedPort.tcp(RustFSContainer.Port)
+ )
)
}
@@ -98,10 +96,11 @@ trait MockLakeFS extends ForAllTestContainer with
BeforeAndAfterAll { self: Suit
env = Map(
"LAKEFS_BLOCKSTORE_TYPE" -> "s3",
"LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE" -> "true",
- "LAKEFS_BLOCKSTORE_S3_ENDPOINT" ->
s"http://${minio.container.getNetworkAliases.get(0)}:9000",
- "LAKEFS_BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT" ->
s"http://localhost:$minioHostPort";,
- "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID" -> "texera_minio",
- "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY" -> "password",
+ "LAKEFS_BLOCKSTORE_S3_ENDPOINT" ->
s"http://${objectStore.container.getNetworkAliases
+ .get(0)}:${RustFSContainer.Port}",
+ "LAKEFS_BLOCKSTORE_S3_PRE_SIGNED_ENDPOINT" ->
s"http://localhost:$objectStoreHostPort";,
+ "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID" ->
RustFSContainer.DefaultUser,
+ "LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY" ->
RustFSContainer.DefaultPassword,
"LAKEFS_AUTH_ENCRYPT_SECRET_KEY" -> "random_string_for_lakefs",
"LAKEFS_LOGGING_LEVEL" -> "INFO",
"LAKEFS_STATS_ENABLED" -> "1",
@@ -114,10 +113,11 @@ trait MockLakeFS extends ForAllTestContainer with
BeforeAndAfterAll { self: Suit
)
lakefs.container.withNetwork(network)
- override val container = MultipleContainers(postgres, minio, lakefs)
+ override val container = MultipleContainers(postgres, objectStore, lakefs)
def lakefsBaseUrl: String =
s"http://${lakefs.host}:${lakefs.mappedPort(8000)}"
- def minioEndpoint: String = s"http://${minio.host}:${minio.mappedPort(9000)}"
+ def objectStoreEndpoint: String =
+
s"http://${objectStore.host}:${objectStore.mappedPort(RustFSContainer.Port)}"
def lakefsApiBasePath: String = s"$lakefsBaseUrl/api/v1"
// ---- Clients (lazy so they initialize after containers are started) ----
@@ -134,20 +134,20 @@ trait MockLakeFS extends ForAllTestContainer with
BeforeAndAfterAll { self: Suit
lazy val repositoriesApi: RepositoriesApi = new
RepositoriesApi(lakefsApiClient)
/**
- * S3 client instance for testing pointed at MinIO.
+ * S3 client instance for testing pointed at RustFS.
*
* Notes:
- * - Region can be any value for MinIO, but MUST match what your signing
expects.
- * so we use that.
+ * - The region MUST match RUSTFS_REGION: it is part of the SigV4
credential scope.
* - Path-style is important: http://host:port/bucket/key
*/
lazy val s3Client: S3Client = {
//Temporal credentials for testing purposes only
- val creds = AwsBasicCredentials.create("texera_minio", "password")
+ val creds =
+ AwsBasicCredentials.create(RustFSContainer.DefaultUser,
RustFSContainer.DefaultPassword)
S3Client
.builder()
.endpointOverride(URI.create(StorageConfig.s3Endpoint)) // set in
afterStart()
- .region(Region.US_WEST_2) // Required for `.build()`; not important in
this test config.
+ .region(Region.US_WEST_2) // must match RustFSContainer's RUSTFS_REGION
.credentialsProvider(StaticCredentialsProvider.create(creds))
.serviceConfiguration(S3Configuration.builder().pathStyleAccessEnabled(true).build())
.build()
@@ -172,7 +172,7 @@ trait MockLakeFS extends ForAllTestContainer with
BeforeAndAfterAll { self: Suit
}
// replace storage endpoints in StorageConfig
- StorageConfig.s3Endpoint = minioEndpoint
+ StorageConfig.s3Endpoint = objectStoreEndpoint
StorageConfig.lakefsEndpoint = lakefsApiBasePath
// create S3 bucket used by lakeFS in tests
diff --git
a/file-service/src/test/scala/org/apache/texera/service/resource/ModelDownloadResourceSpec.scala
b/file-service/src/test/scala/org/apache/texera/service/resource/ModelDownloadResourceSpec.scala
index 9ee52955f8..758c54306f 100644
---
a/file-service/src/test/scala/org/apache/texera/service/resource/ModelDownloadResourceSpec.scala
+++
b/file-service/src/test/scala/org/apache/texera/service/resource/ModelDownloadResourceSpec.scala
@@ -118,7 +118,7 @@ class ModelDownloadResourceSpec
private def presignedUrlOf(response: Response): String =
response.getEntity.asInstanceOf[Map[String, String]]("presignedUrl")
- /** MinIO is bound to a fixed host port precisely so presigned URLs resolve
here. */
+ /** RustFS is bound to a fixed host port precisely so presigned URLs resolve
here. */
private def fetch(url: String): Array[Byte] = {
val stream = new java.net.URL(url).openStream()
try stream.readAllBytes()
diff --git
a/frontend/src/app/dashboard/service/user/model/model.service.spec.ts
b/frontend/src/app/dashboard/service/user/model/model.service.spec.ts
index d764bebb59..52b12be3a1 100644
--- a/frontend/src/app/dashboard/service/user/model/model.service.spec.ts
+++ b/frontend/src/app/dashboard/service/user/model/model.service.spec.ts
@@ -206,8 +206,8 @@ describe("ModelService", () => {
const presign = http.expectOne(
`${API}/model/presign-download?filePath=${encodeURIComponent("/model/a/m/v1/model.pt")}`
);
- presign.flush({ presignedUrl: "http://minio/model.pt"; });
- http.expectOne("http://minio/model.pt";).flush(blob);
+ presign.flush({ presignedUrl: "http://rustfs/model.pt"; });
+ http.expectOne("http://rustfs/model.pt";).flush(blob);
expect(await pending).toEqual(blob);
});
@@ -216,8 +216,8 @@ describe("ModelService", () => {
service.retrieveModelVersionSingleFile("/model/a/m/v1/model.pt",
false).subscribe();
http
.expectOne(`${API}/model/public-presign-download?filePath=${encodeURIComponent("/model/a/m/v1/model.pt")}`)
- .flush({ presignedUrl: "http://minio/model.pt"; });
- http.expectOne("http://minio/model.pt";).flush(new Blob());
+ .flush({ presignedUrl: "http://rustfs/model.pt"; });
+ http.expectOne("http://rustfs/model.pt";).flush(new Blob());
});
it("reads the presigned cover url, which is null for a model without one",
async () => {
