This is an automated email from the ASF dual-hosted git repository.
hello-stephen pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b3b1eab90ca [feat](docker) add an all-in-one image for 4.1 integration
testing (#66983)
b3b1eab90ca is described below
commit b3b1eab90ca567246d0564452e325d6e19688821
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Thu Aug 20 15:57:41 2026 +0800
[feat](docker) add an all-in-one image for 4.1 integration testing (#66983)
## Proposed changes
Downstream projects that want to run integration tests against Doris
need one
container they can start and connect to, not a compose file with an FE,
a BE
and a wait loop. This adds that image for the **4.1 release line** under
`docker/runtime/all-in-one/4.1/`.
It also removes the old `docker/runtime/all-in-one/` recipe, which no
longer
worked: it pinned JDK 8 (so 3.0+ would not start), carried a literal
`apache-doris-x.x.x-bin-` placeholder that had to be hand-edited, and
laid its
`resource/` tree out incompatibly with what `docker-build.sh` produces.
Its
entrypoint also called `stop_fe.sh` / `stop_be.sh`, which cannot work
under
`--console` (neither leaves a usable pid file).
### Assembly, not repackaging
`fe/` and `be/` are copied straight out of the official
`apache/doris:fe-<v>`
and `:be-<v>` images, so a new Doris release needs no work here. buildx
resolves those per target platform, which is what lets one tag be
multi-arch
without staging per-arch directories by hand. A locally built `./output`
and an
extracted release tarball are accepted too; all three normalise into the
same
artifact stage.
### Two tags
| tag | covers | size |
|---|---|---|
| `all-in-one-<v>` | internal tables, Hive, Iceberg (incl. system
tables), Paimon, JDBC catalogs, external-table writeback, Java UDF |
2.46 GB |
| `all-in-one-<v>-full` | the above plus Hudi, Trino connector,
MaxCompute | 2.99 GB |
against 4.9 GB for the same payload untouched.
The largest single saving is not a connector. `apache/doris:be-4.1.3`
ships
`doris_be` with debug info; `strip --strip-debug` takes it from **2213
MB to
430 MB** while keeping `.symtab`, so crash backtraces still resolve
function
names and only lose file:line. Build with `--strip none` to keep the
binary
byte-identical to the official image.
Both the strip and the pruning happen in the artifact stage. Doing
either after
the final `COPY` would only add a whiteout layer and the bytes would
still ship.
### What must never be pruned
`resource/prune.txt` is the only file that needs to track a release, and
it
records the three directories that look like external-table extras but
are the
JNI baseline every such read goes through:
- `be/lib/java_extensions/preload-extensions` — no Java sources at all;
it is
the parquet-hadoop-bundle / hadoop-common / libthrift / arrow / AWS SDK
dependency bundle. Preloaded by `bin/start_be.sh`, and
`DORIS_PRELOAD_JAR`
must stay first on the classpath.
- `be/lib/java_extensions/java-udf` — preloaded alongside it.
- `be/lib/hadoop_hdfs` — the JVM side of libhdfs (157 jars). libhdfs is
linked
statically into `doris_be` in 4.1, hence the empty `native/`, but it is
still
a JNI wrapper and cannot open an HDFS file without them.
FE is left alone: branch-4.1 has no FE plugin split, so all 634 jars in
`fe/lib` are on the startup classpath.
### Startup is a sequence, not a sleep
FE → poll until ready → BE → register the backend → poll until FE
reports it
alive → flag ready. FE readiness is a metadata query, deliberately
**not**
`select 1`: Nereids picks a backend as the scan node for that, which
cannot
succeed before the BE the entrypoint has not started yet.
`HEALTHCHECK` asks FE's `/api/health`, whose `online_backend_num`
answers both
"is FE ready" and "is the BE alive" in one request, so downstream can
use
`depends_on: condition: service_healthy` instead of sleeping.
If either process exits, the container exits non-zero — a dead FE
becomes a
failed job rather than a job timeout. Shutdown signals process groups,
since
`--console` leaves no usable pid file for the stop scripts.
### Usage
```yaml
services:
doris:
image: apache/doris:all-in-one-4.1.3
ports: ["9030:9030", "8030:8030", "8040:8040"]
integration-test:
image: my-project-tests:latest
depends_on:
doris:
condition: service_healthy
```
## Testing
`resource/smoke-test.sh <tag> [base|full]` starts the image, waits for
`healthy`, asserts the JNI baseline and the flavor payload, then runs
create / insert / stream load / aggregate / schema change. Everything
goes
through `docker exec`, so the host needs only docker.
Both tags come up healthy in ~17 s and pass on **arm64**. Multi-arch
tags were
built and verified locally (one index, `linux/amd64` + `linux/arm64`).
**Not yet verified, and worth a reviewer's attention:**
1. The **amd64** half is verified structurally only — correct ELF, JDK
path and
payload, and `doris_be --version` byte-identical to the official binary
— but
never actually run. Doris BE does not survive emulation on an Apple
Silicon
host, and neither does the stock `apache/doris:be-4.1.3` amd64 image, so
this
needs a run on real x86 hardware.
2. **Reading a real external table** has not been exercised. The base
tag rests
on Hive/Iceberg data reads needing no format-specific JNI scanner — the
set
of scanner classes BE can construct is closed and neither appears in it
— but
that is still an argument from code, not a test. It needs the fixtures
under
`docker/thirdparties`.
## Notes
- No CI wiring in this PR; `build.sh` is meant to be run by hand for
now.
- The directory targets 4.1 only. Other release lines differ enough in
payload
layout to deserve their own directory rather than a version switch.
- Naming: this uses `all-in-one-<version>` to line up with
`fe-<version>` /
`be-<version>`. The repo already has one `doris-all-in-one-2.1.0` tag
under
the older convention; happy to switch if maintainers prefer it.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
https://claude.ai/code/session_01QhGHox19pMRy2cvs7A6xsq
---------
Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
---
docker/runtime/all-in-one/4.1/Dockerfile | 179 +++++++++++++++
.../runtime/all-in-one/4.1/Dockerfile.dockerignore | 27 +++
docker/runtime/all-in-one/4.1/README.md | 251 +++++++++++++++++++++
docker/runtime/all-in-one/4.1/build.sh | 243 ++++++++++++++++++++
.../all-in-one/4.1/resource/conf/be_ci.conf | 37 +++
.../all-in-one/4.1/resource/conf/fe_ci.conf | 41 ++++
.../runtime/all-in-one/4.1/resource/entrypoint.sh | 194 ++++++++++++++++
.../all-in-one/4.1/resource/health_check.sh | 41 ++++
docker/runtime/all-in-one/4.1/resource/prune.txt | 64 ++++++
.../runtime/all-in-one/4.1/resource/smoke-test.sh | 153 +++++++++++++
docker/runtime/all-in-one/Dockerfile | 54 -----
docker/runtime/all-in-one/resource/entry_point.sh | 188 ---------------
12 files changed, 1230 insertions(+), 242 deletions(-)
diff --git a/docker/runtime/all-in-one/4.1/Dockerfile
b/docker/runtime/all-in-one/4.1/Dockerfile
new file mode 100644
index 00000000000..31c3c829bf8
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/Dockerfile
@@ -0,0 +1,179 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# ============================================================================
+# Apache Doris all-in-one image for integration testing -- branch-4.1 only.
+#
+# A single container running one FE and one BE, sized for use as a test
+# fixture in downstream CI. Build it with ./build.sh; the commands below are
+# what that script ends up running.
+#
+# base flavor (default):
+# docker buildx build --build-arg DORIS_VERSION=4.1.3 \
+# -f docker/runtime/all-in-one/4.1/Dockerfile \
+# -t apache/doris:all-in-one-4.1.3 .
+#
+# full flavor (adds hudi / trino / maxcompute):
+# ... --build-arg FLAVOR=full -t apache/doris:all-in-one-4.1.3-full .
+#
+# The build context is the repository root (so ARTIFACT_SOURCE=local can pick
+# up ./output); Dockerfile.dockerignore narrows it down to a few KB.
+# ============================================================================
+
+# ---- global args: must be declared before the first FROM ----
+ARG DORIS_VERSION=4.1.3
+ARG ARTIFACT_SOURCE=image
+ARG FE_IMAGE=apache/doris:fe-${DORIS_VERSION}
+ARG BE_IMAGE=apache/doris:be-${DORIS_VERSION}
+ARG BASE_IMAGE=ubuntu:22.04
+
+# ============================ artifact sources ==============================
+# Three interchangeable ways to get fe/ and be/ into /artifacts. Everything
+# downstream only knows about /artifacts, never about where it came from.
+
+FROM ${FE_IMAGE} AS fe-src
+FROM ${BE_IMAGE} AS be-src
+
+# strip(1) lives here, so this is ubuntu rather than busybox.
+FROM ${BASE_IMAGE} AS artifacts-base
+RUN apt-get update -y && \
+ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+ -o Acquire::Retries=3 binutils && \
+ rm -rf /var/lib/apt/lists/*
+
+# A. the official component images (default)
+FROM artifacts-base AS artifacts-image
+COPY --from=fe-src /opt/apache-doris/fe /artifacts/fe
+COPY --from=be-src /opt/apache-doris/be /artifacts/be
+
+# B. a locally built ./output, for developers testing their own build
+FROM artifacts-base AS artifacts-local
+ARG LOCAL_OUTPUT=output
+COPY ${LOCAL_OUTPUT}/fe /artifacts/fe
+COPY ${LOCAL_OUTPUT}/be /artifacts/be
+
+# C. an extracted release tarball
+FROM artifacts-base AS artifacts-tarball
+ARG TARBALL_DIR
+COPY ${TARBALL_DIR}/fe /artifacts/fe
+COPY ${TARBALL_DIR}/be /artifacts/be
+
+# ========================= strip + flavor pruning ===========================
+# Both have to happen HERE, before the final COPY. Doing them in the runtime
+# stage would only add a whiteout layer -- the bytes would still ship.
+
+FROM artifacts-${ARTIFACT_SOURCE} AS artifacts
+ARG CTX_PREFIX=docker/runtime/all-in-one/4.1
+ARG FLAVOR=base
+ARG STRIP_BE=debug
+
+COPY ${CTX_PREFIX}/resource/prune.txt /tmp/prune.txt
+
+RUN set -eux; \
+ case "${FLAVOR}" in base|full) ;; *) echo "bad FLAVOR=${FLAVOR}" >&2; exit
1 ;; esac; \
+ # 4.1.3 ships doris_be with debug info: 2213M -> 430M, and --strip-debug
+ # keeps .symtab so crash backtraces still resolve function names.
+ case "${STRIP_BE}" in \
+ debug) strip --strip-debug /artifacts/be/lib/doris_be ;; \
+ full) strip -s /artifacts/be/lib/doris_be ;; \
+ none) : ;; \
+ *) echo "bad STRIP_BE=${STRIP_BE}" >&2; exit 1 ;; \
+ esac; \
+ # "always" rows apply to both flavors, "base" rows only to the base flavor.
+ awk -v f="${FLAVOR}" '$1=="always" || $1==f {print $2}' /tmp/prune.txt \
+ | while IFS= read -r p; do echo "prune: ${p}"; rm -rf "/artifacts/${p}";
done; \
+ rm -f /tmp/prune.txt; \
+ rm -rf /artifacts/fe/log /artifacts/be/log; \
+ mkdir -p /artifacts/fe/log /artifacts/be/log \
+ /artifacts/fe/doris-meta /artifacts/be/storage; \
+ echo "=== artifact size (${FLAVOR}) ==="; du -sh /artifacts/fe
/artifacts/be
+
+# ================================ runtime ===================================
+
+FROM ${BASE_IMAGE} AS runtime
+ARG TARGETARCH
+ARG DORIS_VERSION
+ARG FLAVOR=base
+ARG CTX_PREFIX=docker/runtime/all-in-one/4.1
+ARG JDK_PKG=openjdk-17-jre-headless
+# FE ships -Xmx8192m, far more than a CI runner can spare; BE ships -Xmx2048m
+# for its JNI-side JVM. Both are rewritten in place below.
+ARG FE_HEAP=2048m
+ARG BE_HEAP=1024m
+
+ENV DORIS_HOME=/opt/apache-doris \
+ CI_HOME=/opt/doris-ci \
+ LANG=C.UTF-8 \
+ TZ=UTC
+
+RUN set -eux; \
+ apt-get update -y; \
+ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
+ -o Acquire::Retries=3 \
+ ${JDK_PKG} \
+ mysql-client-core-8.0 \
+ curl \
+ tini \
+ procps \
+ tzdata \
+ ca-certificates; \
+ ln -sfn "/usr/lib/jvm/java-17-openjdk-${TARGETARCH:-amd64}"
/usr/lib/jvm/java; \
+ ln -sf /usr/share/zoneinfo/UTC /etc/localtime; \
+ rm -rf /var/lib/apt/lists/* /usr/share/doc/* /usr/share/man/*
+
+# SKIP_CHECK_ULIMIT: start_be.sh checks vm.max_map_count / swap / ulimit -n and
+# exits 1 when they are too low. vm.max_map_count is not a namespaced sysctl,
so
+# a container cannot fix it -- the check has to be skipped here.
+ENV JAVA_HOME=/usr/lib/jvm/java \
+
PATH=/usr/lib/jvm/java/bin:/opt/apache-doris/fe/bin:/opt/apache-doris/be/bin:$PATH
\
+ SKIP_CHECK_ULIMIT=true
+
+COPY --from=artifacts /artifacts/fe ${DORIS_HOME}/fe
+COPY --from=artifacts /artifacts/be ${DORIS_HOME}/be
+COPY ${CTX_PREFIX}/resource/ ${CI_HOME}/
+
+RUN set -eux; \
+ # Integration-test defaults are appended, never edited in: the upstream
+ # values stay visible above them and the last assignment wins for both the
+ # shell-sourced ALL_CAPS vars and the property-style lowercase keys.
+ cat "${CI_HOME}/conf/fe_ci.conf" >> "${DORIS_HOME}/fe/conf/fe.conf"; \
+ cat "${CI_HOME}/conf/be_ci.conf" >> "${DORIS_HOME}/be/conf/be.conf"; \
+ # The heap lives inside a long JAVA_OPTS_FOR_JDK_17 line that also carries
+ # every --add-opens FE needs on JDK 17. Rewriting just the -Xmx/-Xms tokens
+ # keeps the rest of that line exactly as upstream shipped it.
+ sed -i -E "s/-Xmx[0-9]+[kKmMgG]/-Xmx${FE_HEAP}/g;
s/-Xms[0-9]+[kKmMgG]/-Xms${FE_HEAP}/g" \
+ "${DORIS_HOME}/fe/conf/fe.conf"; \
+ sed -i -E "s/-Xmx[0-9]+[kKmMgG]/-Xmx${BE_HEAP}/g;
s/-Xms[0-9]+[kKmMgG]/-Xms${BE_HEAP}/g" \
+ "${DORIS_HOME}/be/conf/be.conf"; \
+ chmod +x "${CI_HOME}"/*.sh
+
+LABEL org.opencontainers.image.title="Apache Doris all-in-one" \
+ org.opencontainers.image.description="Single-container Doris (FE+BE) for
integration testing" \
+ org.opencontainers.image.version="${DORIS_VERSION}" \
+ org.opencontainers.image.source="https://github.com/apache/doris"; \
+ org.apache.doris.allinone.flavor="${FLAVOR}" \
+ org.apache.doris.allinone.branch="4.1"
+
+# 8030 FE http | 9030 FE mysql | 8040 BE http (stream load) | 9050 BE heartbeat
+EXPOSE 8030 9030 8040 9050
+
+WORKDIR ${DORIS_HOME}
+
+HEALTHCHECK --interval=5s --timeout=5s --start-period=20s --retries=90 \
+ CMD /opt/doris-ci/health_check.sh
+
+ENTRYPOINT ["/usr/bin/tini", "--", "/opt/doris-ci/entrypoint.sh"]
diff --git a/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
b/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
new file mode 100644
index 00000000000..379b506fd84
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/Dockerfile.dockerignore
@@ -0,0 +1,27 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# BuildKit reads this in preference to the repository root .dockerignore when
+# building with -f docker/runtime/all-in-one/4.1/Dockerfile.
+#
+# The context has to be the repository root so that ARTIFACT_SOURCE=local can
+# reach ./output, but nothing else in the tree is needed. Exclude everything,
+# then add back the two paths the build actually reads.
+*
+!docker/runtime/all-in-one/4.1/resource
+!output/fe
+!output/be
diff --git a/docker/runtime/all-in-one/4.1/README.md
b/docker/runtime/all-in-one/4.1/README.md
new file mode 100644
index 00000000000..49d8d1d6888
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/README.md
@@ -0,0 +1,251 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Doris all-in-one image (branch-4.1)
+
+One FE and one BE in a single container, sized to be a test fixture in a
+downstream project's CI. It is assembled from the official `apache/doris:fe-*`
+and `apache/doris:be-*` images, so a new Doris release needs no repackaging of
+anything here.
+
+This directory targets the **4.1 release line only**. Other lines differ enough
+in payload layout to deserve their own directory rather than a version switch.
+
+## Two tags
+
+| tag | covers | size |
+|---|---|---|
+| `apache/doris:all-in-one-<version>` | internal tables, Hive, Iceberg
(including system tables), Paimon, JDBC catalogs, external-table writeback,
Java UDF | 2.46 GB |
+| `apache/doris:all-in-one-<version>-full` | the above plus Hudi, Trino
connector, MaxCompute | 2.99 GB |
+
+Pick `-full` only if the tests touch Hudi, the Trino connector or MaxCompute.
+
+Both come up `healthy` in under 20 seconds. Sizes are the uncompressed layer
+sum measured on 4.1.3/arm64, against 4.9 GB for the same payload untouched.
+`docker image inspect --format '{{.Size}}'` reports 1.61 / 2.06 GiB for them;
+`docker images` can print a much larger figure when the containerd image store
+is enabled, because it adds the compressed blobs to the unpacked snapshot.
+
+## Build
+
+A plain run builds for the host architecture only. Multi-arch needs an explicit
+`--platform`; the header the script prints says which it is doing.
+
+```shell
+# both tags, from the official 4.1.3 component images, host architecture only
+./build.sh -v 4.1.3
+
+# base tag only, then smoke test it
+./build.sh -v 4.1.3 -f base -t
+
+# from a locally built ./output instead
+./build.sh -v dev -s local
+```
+
+`./build.sh --help` lists the rest. The build context is the repository root,
+narrowed to a few KB by `Dockerfile.dockerignore`; run the script from
anywhere.
+
+## Multi-architecture
+
+One tag can serve both architectures, the way `apache/doris:fe-4.1.3` does: the
+tag points at an OCI image index listing one manifest per platform, and a pull
+picks the matching entry.
+
+```
+apache/doris:all-in-one-4.1.3 -> index
+ |- linux/amd64
+ \- linux/arm64
+```
+
+Nothing in the Dockerfile is architecture-specific beyond one JDK symlink;
+`COPY --from=apache/doris:be-4.1.3` resolves to the target platform's variant
on
+its own, so no per-arch directories have to be staged by hand.
+
+**Both architectures on one host.** Simple, but slow for whichever architecture
+is foreign to the machine:
+
+```shell
+./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+```
+
+Keeping the result locally instead of pushing works only with the containerd
+image store, which is where Docker has somewhere to put a second architecture.
+
+**One architecture per host, joined afterwards.** No emulation, so this is much
+faster and is the sane choice for a release:
+
+```shell
+# on an x86 host
+./build.sh -v 4.1.3 --platform linux/amd64 --push -i myrepo/doris # then
retag ...-amd64
+# on an arm host
+./build.sh -v 4.1.3 --platform linux/arm64 --push -i myrepo/doris # then
retag ...-arm64
+# anywhere
+docker buildx imagetools create -t myrepo/doris:all-in-one-4.1.3 \
+ myrepo/doris:all-in-one-4.1.3-amd64 myrepo/doris:all-in-one-4.1.3-arm64
+```
+
+**What emulation costs.** Measured on an Apple Silicon host building amd64:
+the two `apt-get install` layers take about 18 minutes between them, plus a
+one-off pull of that architecture's `be` image. Everything else is cheap --
+the `COPY --from` steps are plain file copies at 8 seconds total and `strip`
+takes under 2 seconds even emulated. If that ever needs to come down, the
+runtime `apt` is the only target worth attacking, e.g. by basing the runtime
+stage on a multi-arch image that already carries a JDK.
+
+Note that a Doris BE built for a foreign architecture generally will not *run*
+under emulation -- the official `apache/doris:be-4.1.3` amd64 image segfaults
on
+an Apple Silicon host just as this image does. Build for the other architecture
+freely; test it on real hardware.
+
+## Use
+
+The image ships a `HEALTHCHECK`, so nothing downstream has to sleep.
+
+```yaml
+# docker compose
+services:
+ doris:
+ image: apache/doris:all-in-one-4.1.3
+ ports: ["9030:9030", "8030:8030", "8040:8040"]
+
+ integration-test:
+ image: my-project-tests:latest
+ depends_on:
+ doris:
+ condition: service_healthy
+```
+
+```yaml
+# GitHub Actions -- service containers are started and waited on automatically
+services:
+ doris:
+ image: apache/doris:all-in-one-4.1.3
+ ports: ['9030:9030', '8030:8030', '8040:8040']
+```
+
+| | |
+|---|---|
+| MySQL protocol | `9030`, user `root`, no password |
+| FE HTTP | `8030`, `/api/health` needs no auth |
+| BE HTTP | `8040`, stream load endpoint |
+| readiness | docker health status turns `healthy` once FE is up and the BE is
registered and alive |
+| replicas | forced to 1, so `CREATE TABLE` needs no `replication_num` |
+| persistence | none by default; mount `/opt/apache-doris/fe/doris-meta` and
`/opt/apache-doris/be/storage` to keep state, the startup path is idempotent |
+| config override | `-e FE_CONFIG_EXTRA=...` / `-e BE_CONFIG_EXTRA=...`,
appended to the respective conf at startup |
+| exit behaviour | fail-fast: if FE or BE exits, the container exits non-zero.
`docker stop` shuts BE then FE down gracefully and exits 0 |
+| logs | `docker logs` carries FE's console stream; the full logs live in
`fe/log/fe.log` and `be/log/be.INFO` |
+
+Heap and memory are tuned down for CI runners (FE `-Xmx2048m`, BE JNI heap
+`-Xmx1024m`, BE `mem_limit = 40%`). On a larger machine, raise them with
+`-e BE_CONFIG_EXTRA="mem_limit = 80%"` or rebuild with `--build-arg
FE_HEAP=8192m`.
+
+Heavy workloads may also need `vm.max_map_count` raised **on the host**
+(`sysctl -w vm.max_map_count=2000000`); it is not a namespaced sysctl, so the
+container cannot set it and `start_be.sh`'s check for it is skipped.
+
+## How the two tags differ
+
+BE loads JNI scanners by enumerating the directories under
+`be/lib/java_extensions/` (`ScannerLoader.loadAllScannerJars`, called from
+`be/src/util/jni-util.cpp` when the BE JVM starts). There is no list and no
+config key, so a tag supports exactly the formats whose directory it ships.
+
+The set of scanner classes BE can construct is closed — every one of them is
+named as a string literal under `be/src/format_v2/jni/` and
+`be/src/format/table/`:
+
+| class | directory | tag |
+|---|---|---|
+| `PaimonJniScanner` | `paimon-scanner` | both |
+| `IcebergSysTableJniScanner` | `iceberg-metadata-scanner` | both |
+| `JdbcJniScanner` | `jdbc-scanner` | both |
+| (`JniWriter`) | `java-writer` | both |
+| `HadoopHudiJniScanner` | `hadoop-hudi-scanner` | `-full` |
+| `TrinoConnectorJniScanner` | `trino-connector-scanner` | `-full` |
+| `MaxComputeJniScanner` | `max-compute-connector` | `-full` |
+
+Hive and Iceberg **data** reads are not in that table: their column decoding
+runs in BE's native parquet/orc reader. They still go through JNI for file
+system and catalog access, which is what the next section is about.
+
+## The JNI baseline — do not prune these
+
+Three directories look like external-table extras and are not:
+
+- **`be/lib/java_extensions/preload-extensions`** (254 MB) — no Java sources at
+ all, just a dependency bundle: parquet-hadoop-bundle, hadoop-common,
+ hadoop-cos, libthrift, arrow, and the AWS SDK including
+ `s3-tables-catalog-for-iceberg`. Every external-table read passes through it.
+ `bin/start_be.sh` preloads it and requires `DORIS_PRELOAD_JAR` to stay first
+ on the classpath.
+- **`be/lib/java_extensions/java-udf`** (237 MB) — preloaded alongside it.
+- **`be/lib/hadoop_hdfs`** (99 MB) — the JVM side of libhdfs, 157 jars. In 4.1
+ libhdfs is linked statically into `doris_be` (hence the empty `native/`
+ directory), but it is still a JNI wrapper and cannot open an HDFS file
+ without them.
+
+Together they are ~590 MB, a quarter of the base tag. `resource/prune.txt`
+deliberately omits them and says so.
+
+## What the build actually removes
+
+| | base | -full |
+|---|---|---|
+| `strip --strip-debug` on `doris_be` | 2213 MB → 430 MB (450 MB on arm64) |
same |
+| `be/lib/meta_tool`, `be/lib/cdc_client`, `fe/arthas` | removed | removed |
+| hudi / trino / maxcompute scanners | removed | kept |
+
+Stripping is the single largest saving — more than every connector combined —
+and `--strip-debug` keeps `.symtab`, so crash backtraces still resolve function
+names and only lose file:line. Build with `--strip none` to keep the binary
+byte-identical to the official image.
+
+Both the strip and the pruning happen in the artifact stage, before the final
+`COPY`. Doing either in the runtime stage would only add a whiteout layer and
+the bytes would still ship.
+
+FE is left alone: branch-4.1 has no FE plugin split, so all 634 jars in
+`fe/lib` are on the startup classpath and pruning by jar name would be fragile
+for little gain.
+
+## Verifying a change to `resource/prune.txt`
+
+`resource/smoke-test.sh <image:tag> [base|full]` starts the image, waits for
+`healthy`, asserts the JNI baseline and the flavor payload are what they should
+be, then exercises create / insert / stream load / aggregate / schema change.
+It runs everything through `docker exec`, so the host needs only docker.
+
+It does **not** read a real external table. To check that end of things, point
+the built image at the fixtures under `docker/thirdparties` and run an Iceberg
+or Hive query by hand.
+
+## Layout
+
+```
+4.1/
+├── Dockerfile three artifact sources -> strip + prune ->
runtime
+├── Dockerfile.dockerignore keeps the repo-root context to a few KB
+├── build.sh the only entry point you need
+└── resource/
+ ├── entrypoint.sh FE -> readiness -> BE -> register -> fail-fast
wait
+ ├── health_check.sh backs HEALTHCHECK
+ ├── smoke-test.sh guards prune.txt
+ ├── prune.txt what each flavor drops, and what must never be
dropped
+ └── conf/{fe_ci.conf,be_ci.conf} appended to the upstream conf at build
time
+```
diff --git a/docker/runtime/all-in-one/4.1/build.sh
b/docker/runtime/all-in-one/4.1/build.sh
new file mode 100755
index 00000000000..55f509e1c85
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/build.sh
@@ -0,0 +1,243 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Builds the Doris 4.1.x all-in-one images locally.
+
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "${HERE}/../../../.." && pwd)"
+DOCKERFILE="docker/runtime/all-in-one/4.1/Dockerfile"
+
+IMAGE="${IMAGE:-apache/doris}"
+VERSION=""
+SOURCE="image"
+FLAVORS=""
+STRIP_BE="debug"
+PLATFORM=""
+LOCAL_OUTPUT="output"
+TARBALL_DIR=""
+RUN_TEST=false
+NO_CACHE=false
+PUSH=false
+
+usage() {
+ cat <<'USAGE'
+Usage: ./build.sh -v <doris-version> [options]
+
+Builds one container image per flavor. Both flavors share every layer up to the
+artifact stage, so building the second one after the first is cheap.
+
+ base internal tables, Hive, Iceberg (incl. system tables), Paimon,
+ JDBC catalogs, external-table writeback, Java UDF -> :all-in-one-<v>
+ full the above plus Hudi, Trino connector, MaxCompute ->
:all-in-one-<v>-full
+
+Options:
+ -v, --version <v> Doris version, e.g. 4.1.3. Required.
+ -f, --flavor <f> base | full | both (default: both)
+ -s, --source <s> image | local | tarball (default: image)
+ image -> apache/doris:fe-<v> and :be-<v>
+ local -> ./output/{fe,be} from a local build
+ tarball -> --tarball-dir
+ --tarball-dir <d> Extracted release package holding fe/ and be/,
+ as a path relative to the repository root.
+ --local-output <d> Override the ./output path for --source local.
+ --strip <mode> debug | full | none (default: debug)
+ debug -> strip --strip-debug, keeps .symtab
+ full -> strip -s
+ none -> ship doris_be as-is (+1.8 GB)
+ --platform <p> Target platform(s), e.g. linux/amd64 or
+ linux/amd64,linux/arm64. Comma-separated values produce
+ one multi-arch tag (an OCI image index) that resolves
to
+ the right image per host, the way apache/doris:fe-*
does.
+ Keeping a multi-platform result locally needs the
+ containerd image store; otherwise pass --push.
+ --push Push instead of loading into the local image store.
+ -i, --image <name> Image name without tag (default: apache/doris)
+ --no-cache Pass --no-cache to docker build.
+ -t, --test Run resource/smoke-test.sh against each image built.
+ -h, --help This message.
+
+Examples:
+ ./build.sh -v 4.1.3 # both flavors from the official
images
+ ./build.sh -v 4.1.3 -f base -t # base only, then smoke test it
+ ./build.sh -v dev -s local -f full # from a local ./output
+ ./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+ # one multi-arch tag for both
+
+Building a foreign architecture goes through emulation, and the two apt layers
+dominate: roughly 18 minutes per foreign arch on an Apple Silicon host, plus
the
+one-off pull of that architecture's be image. Where both architectures matter,
+building each one natively and joining them afterwards is far faster:
+
+ # on an x86 host
+ ./build.sh -v 4.1.3 --push -i myrepo/doris --platform linux/amd64 # tag it
-amd64 by hand
+ # on an arm host
+ ./build.sh -v 4.1.3 --push -i myrepo/doris --platform linux/arm64 # tag it
-arm64 by hand
+ # then, anywhere
+ docker buildx imagetools create -t myrepo/doris:all-in-one-4.1.3 \
+ myrepo/doris:all-in-one-4.1.3-amd64 myrepo/doris:all-in-one-4.1.3-arm64
+USAGE
+}
+
+while [[ $# -gt 0 ]]; do
+ case "$1" in
+ -v|--version) VERSION="$2"; shift 2 ;;
+ -f|--flavor) FLAVORS="$2"; shift 2 ;;
+ -s|--source) SOURCE="$2"; shift 2 ;;
+ --tarball-dir) TARBALL_DIR="$2"; shift 2 ;;
+ --local-output) LOCAL_OUTPUT="$2"; shift 2 ;;
+ --strip) STRIP_BE="$2"; shift 2 ;;
+ --platform) PLATFORM="$2"; shift 2 ;;
+ -i|--image) IMAGE="$2"; shift 2 ;;
+ --no-cache) NO_CACHE=true; shift ;;
+ --push) PUSH=true; shift ;;
+ -t|--test) RUN_TEST=true; shift ;;
+ -h|--help) usage; exit 0 ;;
+ *) echo "unknown option: $1" >&2; usage; exit 1 ;;
+ esac
+done
+
+[[ -n "${VERSION}" ]] || { echo "error: -v/--version is required" >&2; usage;
exit 1; }
+
+case "${FLAVORS:-both}" in
+ both|"") FLAVORS="base full" ;;
+ base) FLAVORS="base" ;;
+ full) FLAVORS="full" ;;
+ *) echo "error: bad --flavor '${FLAVORS}', expected base|full|both" >&2;
exit 1 ;;
+esac
+
+case "${SOURCE}" in
+ image|local|tarball) ;;
+ *) echo "error: bad --source '${SOURCE}'" >&2; exit 1 ;;
+esac
+case "${STRIP_BE}" in
+ debug|full|none) ;;
+ *) echo "error: bad --strip '${STRIP_BE}'" >&2; exit 1 ;;
+esac
+
+if [[ "${SOURCE}" == tarball && -z "${TARBALL_DIR}" ]]; then
+ echo "error: --source tarball needs --tarball-dir" >&2; exit 1
+fi
+if [[ "${SOURCE}" == local && ! -d "${REPO_ROOT}/${LOCAL_OUTPUT}/be" ]]; then
+ echo "error: ${REPO_ROOT}/${LOCAL_OUTPUT}/be not found; build Doris first"
>&2; exit 1
+fi
+
+command -v docker >/dev/null || { echo "error: docker not found" >&2; exit 1; }
+
+# A comma in --platform means one tag carrying an OCI image index. Docker can
+# only hold that locally with the containerd image store; the classic store has
+# nowhere to put a second architecture, so the build has to go straight to a
+# registry.
+MULTI_PLATFORM=false
+case "${PLATFORM}" in *,*) MULTI_PLATFORM=true ;; esac
+if [[ "${MULTI_PLATFORM}" == true && "${PUSH}" != true ]]; then
+ if ! docker info --format '{{range .DriverStatus}}{{.}}{{end}}'
2>/dev/null | grep -q containerd; then
+ echo "error: a multi-platform build needs --push, or the containerd
image store" >&2
+ echo " (Docker Desktop: Settings > General > Use containerd for
pulling and storing images)" >&2
+ exit 1
+ fi
+fi
+if [[ "${MULTI_PLATFORM}" == true && "${RUN_TEST}" == true ]]; then
+ echo "error: --test cannot run against a multi-platform tag; build one
platform at a time" >&2
+ exit 1
+fi
+
+builder="docker buildx build"
+docker buildx version >/dev/null 2>&1 || builder="docker build"
+
+# Say which platforms are being built. Leaving --platform unset means the host
+# architecture only, which is easy to mistake for a multi-arch build.
+if [[ -n "${PLATFORM}" ]]; then
+ platform_note="${PLATFORM}"
+else
+ platform_note="$(docker version --format '{{.Server.Os}}/{{.Server.Arch}}'
2>/dev/null || echo host)"
+ platform_note="${platform_note} (host only -- pass --platform for
multi-arch)"
+fi
+
+echo "repository root : ${REPO_ROOT}"
+echo "doris version : ${VERSION}"
+echo "artifact source : ${SOURCE}"
+echo "strip mode : ${STRIP_BE}"
+echo "flavors : ${FLAVORS}"
+echo "platform(s) : ${platform_note}"
+echo "output : $([[ "${PUSH}" == true ]] && echo 'push to registry'
|| echo 'load into local image store')"
+echo
+
+built_tags=()
+
+for flavor in ${FLAVORS}; do
+ suffix=""
+ [[ "${flavor}" == full ]] && suffix="-full"
+ tag="${IMAGE}:all-in-one-${VERSION}${suffix}"
+
+ args=(
+ --build-arg "DORIS_VERSION=${VERSION}"
+ --build-arg "ARTIFACT_SOURCE=${SOURCE}"
+ --build-arg "FLAVOR=${flavor}"
+ --build-arg "STRIP_BE=${STRIP_BE}"
+ --build-arg "LOCAL_OUTPUT=${LOCAL_OUTPUT}"
+ )
+ [[ -n "${TARBALL_DIR}" ]] && args+=(--build-arg
"TARBALL_DIR=${TARBALL_DIR}")
+ [[ -n "${PLATFORM}" ]] && args+=(--platform "${PLATFORM}")
+ [[ "${NO_CACHE}" == true ]] && args+=(--no-cache)
+ if [[ "${PUSH}" == true ]]; then
+ args+=(--push)
+ elif [[ "${builder}" == "docker buildx build" ]]; then
+ # buildx leaves the result in the build cache by default; --load puts
it
+ # in the local image store where docker run can see it.
+ args+=(--load)
+ fi
+
+ echo ">>> building ${tag} (flavor=${flavor})"
+ DOCKER_BUILDKIT=1 ${builder} "${args[@]}" \
+ -f "${DOCKERFILE}" -t "${tag}" "${REPO_ROOT}"
+
+ built_tags+=("${tag}:${flavor}")
+ echo
+done
+
+echo "=== built images ==="
+if [[ "${PUSH}" == true ]]; then
+ for entry in "${built_tags[@]}"; do
+ tag="${entry%:*}"
+ echo " pushed ${tag}"
+ docker buildx imagetools inspect "${tag}" 2>/dev/null \
+ | grep -E "^(Name|MediaType)|Platform:" | sed 's/^/ /' || true
+ done
+ exit 0
+fi
+# Reported by docker image inspect. Note that `docker images` can print a much
+# larger number when the containerd image store is enabled: it adds the
+# compressed blobs to the unpacked snapshot instead of reporting one of them.
+for entry in "${built_tags[@]}"; do
+ tag="${entry%:*}"
+ bytes="$(docker image inspect "${tag}" --format '{{.Size}}' 2>/dev/null ||
echo 0)"
+ awk -v t="${tag}" -v b="${bytes}" \
+ 'BEGIN { printf " %-42s %.2f GiB\n", t, b / 1073741824 }'
+done
+
+if [[ "${RUN_TEST}" == true ]]; then
+ for entry in "${built_tags[@]}"; do
+ tag="${entry%:*}"
+ flavor="${entry##*:}"
+ echo
+ echo ">>> smoke test ${tag} (${flavor})"
+ "${HERE}/resource/smoke-test.sh" "${tag}" "${flavor}"
+ done
+fi
diff --git a/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
b/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
new file mode 100644
index 00000000000..8e26b7cb095
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/conf/be_ci.conf
@@ -0,0 +1,37 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#### appended by the all-in-one image -- integration-test defaults ####
+#
+# be.conf is read the same way fe.conf is, and the last assignment wins.
+# The JNI-side heap is not set here; the Dockerfile rewrites -Xmx/-Xms inside
+# the upstream JAVA_OPTS_FOR_JDK_17 line.
+
+priority_networks = 127.0.0.1/32
+
+# Upstream default is 90%, measured against the cgroup limit when there is one
+# and against host memory when there is not. An unconstrained container on a
+# 7 GB CI runner would leave nothing for FE, so cap it well below that.
+# Raise it with -e BE_CONFIG_EXTRA="mem_limit = 80%" when the box is bigger.
+mem_limit = 40%
+
+# A test fixture does not need compaction throughput, and these threads are
+# per disk, so the defaults are pure overhead here.
+base_compaction_num_threads_per_disk = 1
+cumulative_compaction_num_threads_per_disk = 1
+
+sys_log_roll_num = 2
diff --git a/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
b/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
new file mode 100644
index 00000000000..374fe60d120
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/conf/fe_ci.conf
@@ -0,0 +1,41 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+#### appended by the all-in-one image -- integration-test defaults ####
+#
+# fe.conf is consumed twice: bin/start_fe.sh exports the ALL_CAPS keys as
+# environment variables, and FE parses the whole file as properties. Both take
+# the last assignment, so appending overrides without editing anything above.
+#
+# The heap is not set here; the Dockerfile rewrites the -Xmx/-Xms tokens inside
+# the upstream JAVA_OPTS_FOR_JDK_17 line so the --add-opens flags stay intact.
+
+# One BE means one replica. Saves every downstream CREATE TABLE from having to
+# spell out replication_num=1. Config.java describes this as test-environment
+# only, which is exactly what this image is.
+force_olap_table_replication_num = 1
+
+# Everything lives on the container loopback. Pinning IP mode as well keeps a
+# restart under a different container hostname from invalidating the metadata.
+priority_networks = 127.0.0.1/32
+enable_fqdn_mode = false
+
+# Nothing to balance across a single backend.
+disable_balance = true
+
+# Keep the log directory bounded over a long CI run.
+sys_log_roll_num = 2
diff --git a/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
b/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
new file mode 100755
index 00000000000..1ce68ee0efa
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/entrypoint.sh
@@ -0,0 +1,194 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Brings up one FE and one BE inside a single container and keeps them there.
+#
+# Fail-fast by design: if either process exits, so does the container, with a
+# non-zero status. A test fixture that quietly restarts a dead FE turns a
+# two-second failure into a job timeout.
+
+set -Eeuo pipefail
+
+# Job control, so each child lands in its own process group. This is what makes
+# shutdown work: start_fe.sh / start_be.sh run the real java / doris_be process
+# in the foreground and do not forward signals, and neither writes a usable pid
+# file under --console, so stop_fe.sh / stop_be.sh cannot help us. Signalling
+# the whole group reaches the actual process.
+set -m
+
+DORIS_HOME="${DORIS_HOME:-/opt/apache-doris}"
+FE_HOME="${DORIS_HOME}/fe"
+BE_HOME="${DORIS_HOME}/be"
+READY_FLAG="${DORIS_HOME}/.ready"
+
+HOST=127.0.0.1
+FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
+FE_QUERY_PORT="${FE_QUERY_PORT:-9030}"
+BE_HEARTBEAT_PORT="${BE_HEARTBEAT_PORT:-9050}"
+START_TIMEOUT="${START_TIMEOUT:-300}"
+STOP_TIMEOUT="${STOP_TIMEOUT:-30}"
+
+FE_PID=
+BE_PID=
+
+log() { printf '%s [%-5s] [entrypoint] %s\n' "$(date -Iseconds)" "$1"
"${*:2}"; }
+info() { log INFO "$@"; }
+warn() { log WARN "$@" >&2; }
+die() { log ERROR "$@" >&2; exit 1; }
+
+sql() {
+ mysql -uroot -h"${HOST}" -P"${FE_QUERY_PORT}" -N --batch
--connect-timeout=2 -e "$1" 2>/dev/null
+}
+
+fe_health() {
+ # Public endpoint (HealthAction): 503 until FE is ready, otherwise a body
+ # carrying online_backend_num. curl -f turns the 503 into a non-zero exit.
+ curl -fsS --max-time 4 "http://${HOST}:${FE_HTTP_PORT}/api/health";
2>/dev/null
+}
+
+# ---------------------------------------------------------------- config ----
+# The image already carries the integration-test defaults; this is the runtime
+# escape hatch for downstream projects that need one knob changed.
+apply_env_overrides() {
+ if [[ -n "${FE_CONFIG_EXTRA:-}" ]]; then
+ info "appending FE_CONFIG_EXTRA to fe.conf"
+ printf '\n# --- FE_CONFIG_EXTRA ---\n%s\n' "${FE_CONFIG_EXTRA}"
>>"${FE_HOME}/conf/fe.conf"
+ fi
+ if [[ -n "${BE_CONFIG_EXTRA:-}" ]]; then
+ info "appending BE_CONFIG_EXTRA to be.conf"
+ printf '\n# --- BE_CONFIG_EXTRA ---\n%s\n' "${BE_CONFIG_EXTRA}"
>>"${BE_HOME}/conf/be.conf"
+ fi
+}
+
+# ------------------------------------------------------------------- FE -----
+start_fe() {
+ info "starting FE"
+ "${FE_HOME}/bin/start_fe.sh" --console &
+ FE_PID=$!
+}
+
+wait_fe_ready() {
+ local deadline=$((SECONDS + START_TIMEOUT))
+ while ((SECONDS < deadline)); do
+ kill -0 "${FE_PID}" 2>/dev/null \
+ || die "FE exited during startup, see ${FE_HOME}/log/fe.log"
+ # Two gates: the HTTP endpoint reports FE readiness, and a metadata
+ # query proves the MySQL port is actually serving. It has to be a
+ # metadata query -- `select 1` goes through Nereids, which picks a
+ # backend as its scan node and fails with "No backend available" until
+ # one is registered. The BE is not started yet at this point, so using
+ # it here would deadlock the two waits against each other.
+ if fe_health >/dev/null && sql 'show frontends' | grep -q "${HOST}";
then
+ info "FE is ready after ${SECONDS}s"
+ return 0
+ fi
+ sleep 1
+ done
+ die "FE did not become ready within ${START_TIMEOUT}s, see
${FE_HOME}/log/fe.log"
+}
+
+# ------------------------------------------------------------------- BE -----
+start_be() {
+ info "starting BE"
+ "${BE_HOME}/bin/start_be.sh" --console &
+ BE_PID=$!
+}
+
+register_be() {
+ # Idempotent: a container restarted on a mounted doris-meta already has the
+ # backend in its metadata.
+ if sql 'show backends' | grep -qE
"[[:space:]]${HOST}[[:space:]]+${BE_HEARTBEAT_PORT}[[:space:]]"; then
+ info "backend ${HOST}:${BE_HEARTBEAT_PORT} already registered"
+ else
+ info "registering backend ${HOST}:${BE_HEARTBEAT_PORT}"
+ sql "alter system add backend '${HOST}:${BE_HEARTBEAT_PORT}'" \
+ || die "ALTER SYSTEM ADD BACKEND failed"
+ fi
+}
+
+wait_be_alive() {
+ local deadline=$((SECONDS + START_TIMEOUT))
+ while ((SECONDS < deadline)); do
+ kill -0 "${BE_PID}" 2>/dev/null \
+ || die "BE exited during startup, see ${BE_HOME}/log/be.INFO and
${BE_HOME}/log/be.out"
+ # FE reports how many backends it considers alive, so one request
+ # answers both "is the BE up" and "did FE notice".
+ if fe_health | grep -qE
'"online_backend_num"[[:space:]]*:[[:space:]]*[1-9]'; then
+ info "backend is alive after ${SECONDS}s"
+ return 0
+ fi
+ sleep 1
+ done
+ die "backend did not come alive within ${START_TIMEOUT}s"
+}
+
+# -------------------------------------------------------------- shutdown ----
+stop_one() {
+ local name=$1 pid=$2
+ [[ -n "${pid}" ]] || return 0
+ kill -0 "${pid}" 2>/dev/null || return 0
+ info "stopping ${name}"
+ # Negative pid signals the whole process group, which is where the real
+ # java / doris_be process lives.
+ kill -TERM -"${pid}" 2>/dev/null || kill -TERM "${pid}" 2>/dev/null || true
+ local deadline=$((SECONDS + STOP_TIMEOUT))
+ while ((SECONDS < deadline)); do
+ kill -0 "${pid}" 2>/dev/null || { info "${name} stopped"; return 0; }
+ sleep 1
+ done
+ warn "${name} did not stop within ${STOP_TIMEOUT}s, killing"
+ kill -KILL -"${pid}" 2>/dev/null || kill -KILL "${pid}" 2>/dev/null || true
+}
+
+shutdown() {
+ trap - SIGTERM SIGINT
+ rm -f "${READY_FLAG}"
+ # BE first, so it stops reporting to an FE that is about to go away.
+ stop_one BE "${BE_PID}"
+ stop_one FE "${FE_PID}"
+ exit 0
+}
+
+# ------------------------------------------------------------------ main ----
+main() {
+ trap shutdown SIGTERM SIGINT
+ rm -f "${READY_FLAG}"
+
+ apply_env_overrides
+ start_fe
+ wait_fe_ready
+ start_be
+ register_be
+ wait_be_alive
+
+ touch "${READY_FLAG}"
+ info "cluster is ready -- mysql -uroot -h127.0.0.1 -P${FE_QUERY_PORT}"
+
+ # Park here until something dies. `wait -n` also returns when a trap fires,
+ # so the explicit re-check below distinguishes the two cases.
+ local rc=0
+ wait -n "${FE_PID}" "${BE_PID}" || rc=$?
+ rm -f "${READY_FLAG}"
+
+ if ! kill -0 "${FE_PID}" 2>/dev/null; then
+ die "FE exited (rc=${rc}), see ${FE_HOME}/log/fe.log"
+ fi
+ die "BE exited (rc=${rc}), see ${BE_HOME}/log/be.INFO and
${BE_HOME}/log/be.out"
+}
+
+main "$@"
diff --git a/docker/runtime/all-in-one/4.1/resource/health_check.sh
b/docker/runtime/all-in-one/4.1/resource/health_check.sh
new file mode 100755
index 00000000000..16eaeebc18f
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/health_check.sh
@@ -0,0 +1,41 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# Backs the image HEALTHCHECK. Downstream CI waits on the resulting docker
+# health status instead of sleeping.
+
+set -uo pipefail
+
+DORIS_HOME="${DORIS_HOME:-/opt/apache-doris}"
+HOST=127.0.0.1
+FE_HTTP_PORT="${FE_HTTP_PORT:-8030}"
+BE_HTTP_PORT="${BE_HTTP_PORT:-8040}"
+
+# Bootstrap not finished yet: FE may answer while the backend is still being
+# registered, and a test that connects then sees a cluster with no capacity.
+[[ -f "${DORIS_HOME}/.ready" ]] || exit 1
+
+# FE readiness and backend liveness in one request: HealthAction returns 503
+# until FE is ready, and online_backend_num once it is.
+curl -fsS --max-time 4 "http://${HOST}:${FE_HTTP_PORT}/api/health"; 2>/dev/null
\
+ | grep -qE '"online_backend_num"[[:space:]]*:[[:space:]]*[1-9]' || exit 1
+
+# The BE http port also serves stream load, so check it directly.
+curl -fsS --max-time 4 "http://${HOST}:${BE_HTTP_PORT}/api/health"; >/dev/null
2>&1 || exit 1
+
+exit 0
diff --git a/docker/runtime/all-in-one/4.1/resource/prune.txt
b/docker/runtime/all-in-one/4.1/resource/prune.txt
new file mode 100644
index 00000000000..54be8d707ca
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/prune.txt
@@ -0,0 +1,64 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# What each flavor drops. Format:
+#
+# <flavor> <path relative to /artifacts>
+#
+# always removed from both tags
+# base removed from the base tag only, kept in -full
+#
+# Paths that do not exist are skipped silently, so a directory that a future
+# 4.1.x drops needs no change here. Run smoke-test.sh after editing.
+
+# ---- offline tooling and CDC: no tag needs these ----
+always be/lib/meta_tool
+always be/lib/cdc_client
+always fe/arthas
+
+# ---- format-specific JNI scanners kept only in the -full tag ----
+# Each maps to exactly one class name BE can construct, see the table in
+# README.md. Dropping one disables that format and nothing else.
+base be/lib/java_extensions/hadoop-hudi-scanner
+base be/lib/java_extensions/trino-connector-scanner
+base be/lib/java_extensions/max-compute-connector
+
+# ---- NEVER add these: they are the JNI baseline, not "external table extras"
+#
+# be/lib/java_extensions/preload-extensions
+# Zero Java sources -- a pure dependency bundle (parquet-hadoop-bundle,
+# hadoop-common, hadoop-cos, libthrift, arrow, AWS SDK incl.
+# s3-tables-catalog-for-iceberg). Every external-table read goes through
+# it. Preloaded by bin/start_be.sh, and DORIS_PRELOAD_JAR must stay first
+# on the classpath.
+#
+# be/lib/java_extensions/java-udf
+# Preloaded alongside it by bin/start_be.sh.
+#
+# be/lib/hadoop_hdfs
+# The JVM side of libhdfs (157 jars). libhdfs itself is linked statically
+# into doris_be in 4.1 -- hence the empty native/ directory -- but it is
+# still a JNI wrapper and cannot open an HDFS file without these.
+#
+# be/lib/java_extensions/{paimon-scanner,iceberg-metadata-scanner,
+# jdbc-scanner,java-writer}
+# Deliberately in the base tag: 172M total buys Paimon, Iceberg system
+# tables, JDBC catalogs and external-table writeback.
+#
+# fe/lib
+# branch-4.1 has no FE plugin split; all 634 jars are on the startup
+# classpath. Pruning by jar name is fragile and saves little.
diff --git a/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
new file mode 100755
index 00000000000..1109db5d0fb
--- /dev/null
+++ b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
@@ -0,0 +1,153 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+# smoke-test.sh <image:tag> [base|full]
+#
+# DOCKER_RUN_OPTS passes extra flags to docker run, e.g. to reproduce a
+# memory-constrained CI runner:
+# DOCKER_RUN_OPTS='--memory 6g' smoke-test.sh apache/doris:all-in-one-4.1.3
+#
+# Guards the prune list. Everything runs through `docker exec`, using the
+# client tools already inside the image, so the host needs nothing but docker
+# and no ports have to be published.
+#
+# What it does not cover: real external-table reads. Those need the fixtures
+# under docker/thirdparties and are a separate exercise -- see README.md.
+
+set -euo pipefail
+
+IMAGE=${1:?usage: smoke-test.sh <image:tag> [base|full]}
+FLAVOR=${2:-base}
+WAIT_SECONDS=${WAIT_SECONDS:-300}
+
+CID=
+
+cleanup() {
+ local rc=$?
+ if [[ -n "${CID}" ]]; then
+ if ((rc != 0)); then
+ echo "--- container log (tail) ---" >&2
+ docker logs --tail 120 "${CID}" 2>&1 >&2 || true
+ fi
+ docker rm -f "${CID}" >/dev/null 2>&1 || true
+ fi
+ exit "${rc}"
+}
+trap cleanup EXIT
+
+step() { printf '\n== %s\n' "$*"; }
+fail() { echo "FAIL: $*" >&2; exit 1; }
+
+q() { docker exec "${CID}" mysql -uroot -h127.0.0.1 -P9030 -N --batch -e "$1";
}
+
+in_image() { docker exec "${CID}" test -d "/opt/apache-doris/$1"; }
+
+step "starting ${IMAGE}"
+CID=$(docker run -d ${DOCKER_RUN_OPTS:-} "${IMAGE}")
+# Record what was actually exercised: the image is multi-arch capable and this
+# is the one line that says which variant this run proves anything about.
+echo " image arch : $(docker image inspect "${IMAGE}" --format
'{{.Os}}/{{.Architecture}}')"
+echo " host arch : $(uname -m)"
+echo " container : $(docker exec "${CID}" uname -m 2>/dev/null || echo '?')"
+
+step "waiting for health status"
+deadline=$((SECONDS + WAIT_SECONDS))
+while ((SECONDS < deadline)); do
+ status=$(docker inspect -f '{{.State.Health.Status}}' "${CID}" 2>/dev/null
|| echo missing)
+ [[ "${status}" == healthy ]] && break
+ if [[ "$(docker inspect -f '{{.State.Running}}' "${CID}" 2>/dev/null)" !=
"true" ]]; then
+ fail "container exited before becoming healthy"
+ fi
+ sleep 2
+done
+[[ "$(docker inspect -f '{{.State.Health.Status}}' "${CID}")" == healthy ]] \
+ || fail "never became healthy within ${WAIT_SECONDS}s"
+echo "healthy after ${SECONDS}s"
+
+step "JNI baseline is present"
+# These three are what every external-table read goes through. If a future
+# edit to prune.txt takes one out, this is where it shows up.
+for d in be/lib/java_extensions/preload-extensions \
+ be/lib/java_extensions/java-udf \
+ be/lib/hadoop_hdfs; do
+ in_image "${d}" || fail "missing JNI baseline directory: ${d}"
+ echo " ok ${d}"
+done
+
+step "flavor payload matches ${FLAVOR}"
+# Present in both tags.
+for d in paimon-scanner iceberg-metadata-scanner jdbc-scanner java-writer; do
+ in_image "be/lib/java_extensions/${d}" || fail "missing from base payload:
${d}"
+ echo " ok ${d}"
+done
+# Present only in -full.
+for d in hadoop-hudi-scanner trino-connector-scanner max-compute-connector; do
+ if [[ "${FLAVOR}" == full ]]; then
+ in_image "be/lib/java_extensions/${d}" || fail "full flavor is missing
${d}"
+ echo " ok ${d} (full)"
+ else
+ ! in_image "be/lib/java_extensions/${d}" || fail "base flavor should
not ship ${d}"
+ echo " ok ${d} absent (base)"
+ fi
+done
+# Dropped from both tags.
+for d in be/lib/meta_tool be/lib/cdc_client fe/arthas; do
+ ! in_image "${d}" || fail "${d} should have been pruned"
+ echo " ok ${d} absent"
+done
+
+step "create database and table"
+# No replication_num: force_olap_table_replication_num must supply it.
+q "create database if not exists smoke"
+q "drop table if exists smoke.t"
+q "create table smoke.t (k int, v varchar(32), d date)
+ duplicate key(k) distributed by hash(k) buckets 3"
+[[ "$(q "show create table smoke.t" | grep -c 'replication_allocation')" -ge 1
]] \
+ || echo " note: replication_allocation not shown, continuing"
+
+step "insert and read back"
+q "insert into smoke.t values (1,'a','2026-01-01'),(2,'b','2026-01-02')"
+[[ "$(q 'select count(*) from smoke.t')" == "2" ]] || fail "insert/select
mismatch"
+echo " 2 rows"
+
+step "stream load"
+printf '3,c,2026-01-03\n' >/tmp/smoke_load.csv
+docker cp /tmp/smoke_load.csv "${CID}:/tmp/smoke_load.csv" >/dev/null
+rm -f /tmp/smoke_load.csv
+docker exec "${CID}" curl -sS -u root: \
+ -H "column_separator:," -H "Expect:100-continue" \
+ -T /tmp/smoke_load.csv \
+ -XPUT "http://127.0.0.1:8040/api/smoke/t/_stream_load"; | grep -q
'"Status": *"Success"' \
+ || fail "stream load did not report Success"
+[[ "$(q 'select count(*) from smoke.t')" == "3" ]] || fail "row count after
stream load is wrong"
+echo " 3 rows"
+
+step "aggregation and schema change"
+[[ "$(q "select count(distinct v) from smoke.t")" == "3" ]] || fail
"aggregation result is wrong"
+q "alter table smoke.t add column c1 int default '0'"
+for _ in $(seq 1 60); do
+ [[ "$(q "show alter table column from smoke order by CreateTime desc limit
1" | awk '{print $10}')" == "FINISHED" ]] && break
+ sleep 2
+done
+q "select k, c1 from smoke.t order by k limit 1" >/dev/null || fail "cannot
read after schema change"
+echo " schema change applied"
+
+step "cleanup"
+q "drop database smoke"
+
+printf '\nsmoke test passed: %s (%s)\n' "${IMAGE}" "${FLAVOR}"
diff --git a/docker/runtime/all-in-one/Dockerfile
b/docker/runtime/all-in-one/Dockerfile
deleted file mode 100644
index d0832c49ad6..00000000000
--- a/docker/runtime/all-in-one/Dockerfile
+++ /dev/null
@@ -1,54 +0,0 @@
-#!/bin/bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-# how to use Dockerfile.
-# this is dockerfile for build doris fe image on amd64.
-# when build youself image.
-# 1. pull binary from official website and decompress into resource directory
that the level equals with Dockerfile_be_ubuntu.
-# 2. untar xxxx.tar.gz in resource directory, update the dockerfile field
`apache-doris-xxx`, replace with real version.
-# 3. run commad docker build -t xxx.doris.be:xx -f Dockerfile_be_ubuntu.
-
-# we have support buildx for amd64 and arm64 architecture image build.
-# get the binary from doris github and utar into resource, update the
directory as apache-`version(example:2.0.1)`-bin-`architecture(amd64/arm64)`
mode.
-
-# choose a base image
-FROM ubuntu:22.04
-
-ARG TARGETARCH
-
-RUN apt-get update -y && DEBIAN_FRONTEND=noninteractive apt-get install -y
--no-install-recommends \
- patchelf gdb binutils binutils-common mysql-client \
- curl wget less vim htop iproute2 numactl jq iotop sysstat \
- tcpdump iputils-ping dnsutils strace lsof blktrace tzdata \
- bpfcc-tools linux-headers-realtime linux-tools-realtime silversearcher-ag \
- net-tools openjdk-8-jdk && \
- rm -rf /var/lib/apt/lists/*
-
-# set environment variables
-ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-${TARGETARCH:-amd64} \
- PATH="/opt/apache-doris/fe/bin:/opt/apache-doris/be/bin:${PATH}"
-
-# apache-doris/be from doris release xxxx.tar.gz.please update the version in
follows x.x.x.
-ADD resource/apache-doris-x.x.x-bin-${TARGETARCH:-amd64}/fe
/opt/apache-doris/fe
-ADD resource/apache-doris-x.x.x-bin-${TARGETARCH:-amd64}/be
/opt/apache-doris/be
-
-COPY resource/entry_point.sh /usr/local/bin/
-
-WORKDIR /opt/apache-doris
-
-ENTRYPOINT ["bash","entry_point.sh"]
diff --git a/docker/runtime/all-in-one/resource/entry_point.sh
b/docker/runtime/all-in-one/resource/entry_point.sh
deleted file mode 100644
index 32f38197779..00000000000
--- a/docker/runtime/all-in-one/resource/entry_point.sh
+++ /dev/null
@@ -1,188 +0,0 @@
-#!/bin/bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing,
-# software distributed under the License is distributed on an
-# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-# KIND, either express or implied. See the License for the
-# specific language governing permissions and limitations
-# under the License.
-
-set -eo pipefail
-shopt -s nullglob
-
-DORIS_HOME="/opt/apache-doris"
-
-# Obtain necessary and basic information to complete initialization
-
-# logging functions
-# usage: doris_[note|warn|error] $log_meg
-# ie: doris_warn "task may be risky!"
-# out: 2023-01-08T19:08:16+08:00 [Warn] [Entrypoint]: task may be risky!
-doris_log() {
- local type="$1"
- shift
- # accept argument string or stdin
- local text="$*"
- if [ "$#" -eq 0 ]; then text="$(cat)"; fi
- local dt="$(date -Iseconds)"
- printf '%s [%s] [Entrypoint]: %s\n' "$dt" "$type" "$text"
-}
-doris_note() {
- doris_log Note "$@"
-}
-doris_warn() {
- doris_log Warn "$@" >&2
-}
-doris_error() {
- doris_log ERROR "$@" >&2
- exit 1
-}
-
-# check to see if this file is being run or sourced from another script
-_is_sourced() {
- [ "${#FUNCNAME[@]}" -ge 2 ] &&
- [ "${FUNCNAME[0]}" = '_is_sourced' ] &&
- [ "${FUNCNAME[1]}" = 'source' ]
-}
-
-docker_setup_env() {
- declare -g METADATA_FAILURE_RECOVERY MASTER_FE_IP CURRENT_BE_IP \
- CURRENT_BE_PORT DATABASE_ALREADY_EXISTS
- MASTER_FE_IP="127.0.0.1"
- CURRENT_BE_IP="127.0.0.1"
- CURRENT_BE_PORT=9050
- if [[ $RECOVERY == "true" ]]; then
- METADATA_FAILURE_RECOVERY='true'
- fi
- if [ -d "${DORIS_HOME}/fe/doris-meta/image" ]; then
- DATABASE_ALREADY_EXISTS='true'
- fi
-}
-
-# Execute sql script, passed via stdin
-docker_process_sql() {
- set +e
- mysql -uroot -P9030 -h127.0.0.1 --comments "$@" 2>&1
-}
-
-check_be_status() {
- set +e
- declare -g BE_ALREADY_EXISTS
- for i in {1..300}; do
- if [[ $1 == true ]]; then
- docker_process_sql <<<"show frontends" | grep
"[[:space:]]${MASTER_FE_IP}[[:space:]]"
- else
- docker_process_sql <<<"show backends" | grep
"[[:space:]]${CURRENT_BE_IP}[[:space:]]" | grep
"[[:space:]]${CURRENT_BE_PORT}[[:space:]]" | grep "[[:space:]]true[[:space:]]"
- fi
- be_join_status=$?
- if [[ "${be_join_status}" == 0 ]]; then
- if [[ $1 == true ]]; then
- doris_note "MASTER FE is started!"
- else
- doris_note "EntryPoint Check - Verify that BE is registered to FE
successfully"
- BE_ALREADY_EXISTS=true
- fi
- return
- fi
- if [[ $(( $i % 20 )) == 1 ]]; then
- if [[ $1 == true ]]; then
- doris_note "MASTER FE is not started. retry."
- else
- doris_note "BE is not register. retry."
- fi
- fi
- sleep 1
- done
-}
-
-add_priority_networks() {
- doris_note "add priority_networks ‘127.0.0.1/24’ to
${DORIS_HOME}/be/conf/be.conf"
- echo "priority_networks = 127.0.0.1/24" >>${DORIS_HOME}/be/conf/be.conf
-}
-
-register_be_to_fe() {
- set +e
- # check fe status
- local is_fe_start=false
- if [ -n "$DATABASE_ALREADY_EXISTS" ]; then
- check_be_status
- if [ -n "$BE_ALREADY_EXISTS" ]; then
- doris_warn "Same backend already exists! No need to register again！"
- return
- fi
- fi
- for i in {1..300}; do
- docker_process_sql <<<"alter system add backend
'${CURRENT_BE_IP}:${CURRENT_BE_PORT}'"
- register_be_status=$?
- if [[ $register_be_status == 0 ]]; then
- doris_note "BE successfully registered to FE！"
- is_fe_start=true
- return
- fi
- if [[ $(( $i % 20 )) == 1 ]]; then
- doris_note "Register BE to FE is failed. retry."
- fi
- sleep 1
- done
- if ! [[ $is_fe_start ]]; then
- doris_error "Failed to register BE to FE！Tried 30 times！Maybe FE Start
Failed！"
- fi
-}
-
-start_doris() {
- declare -g child_pid
- if [[ $METADATA_FAILURE_RECOVERY == "true" ]]; then
- doris_warn "Because \$RECOVERY = True, So Doris FE start
metadata_failure_recovery model."
- start_fe.sh --metadata_failure_recovery --console
- else
- doris_note "Start Doris FE."
- {
- set +e
- bash start_fe.sh --console 2>/dev/null
- } &
- fi
- sleep 20
- doris_note "Start Doris BE."
- {
- set +e
- bash start_be.sh --console 2>/dev/null
- } &
- child_pid=$!
-}
-
-stop_doris() {
- doris_note "Container stopped, running stop_fe & stop_be script"
- stop_be.sh
- sleep 10
- stop_fe.sh
-}
-
-_main() {
- trap 'stop_doris' SIGTERM SIGINT
- docker_setup_env
- # Check Already Exists
- if [ -z "$DATABASE_ALREADY_EXISTS" ]; then
- add_priority_networks
- fi
- # Start Doris
- start_doris
- # register BE
- register_be_to_fe
- # keep BE started status
- wait $child_pid
- doris_note "Apache Doris is Start Successfully!"
- exec "$@"
-}
-
-if ! _is_sourced; then
- _main "$@"
-fi
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
