This is an automated email from the ASF dual-hosted git repository.
github-merge-queue[bot] pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new 08a38f1b8c [Docs] Document DolphinScheduler integration with Docker
Compose deployments (#11691)
08a38f1b8c is described below
commit 08a38f1b8c5353228d3e1ba11e2eebb20619844f
Author: jinniiii233 <[email protected]>
AuthorDate: Fri Sep 18 15:02:26 2026 +0000
[Docs] Document DolphinScheduler integration with Docker Compose
deployments (#11691)
---
docs/en/tools/dolphinscheduler-integration.md | 90 +++++++++++++++++++++++++++
docs/sidebars.js | 3 +-
docs/zh/tools/dolphinscheduler-integration.md | 90 +++++++++++++++++++++++++++
3 files changed, 182 insertions(+), 1 deletion(-)
diff --git a/docs/en/tools/dolphinscheduler-integration.md
b/docs/en/tools/dolphinscheduler-integration.md
new file mode 100644
index 0000000000..bcacf3895d
--- /dev/null
+++ b/docs/en/tools/dolphinscheduler-integration.md
@@ -0,0 +1,90 @@
+# Integrating SeaTunnel with DolphinScheduler via Docker Compose
+
+A common question when deploying SeaTunnel through Docker Compose is: which
`SEATUNNEL_HOME` path should DolphinScheduler be configured with so it can
correctly submit SeaTunnel jobs? This document explains the reasoning and
provides a minimal, ready-to-use example.
+
+## 1. What `SEATUNNEL_HOME` Means for Scheduler Integration
+
+DolphinScheduler does not call SeaTunnel over a network API. Instead, it
directly executes SeaTunnel's launch script on the same machine (or container)
it runs on, for example:
+
+```
+${SEATUNNEL_HOME}/bin/seatunnel.sh --config <job_config_path>
+```
+
+This means `SEATUNNEL_HOME` must point to a path **that the DolphinScheduler
process itself can actually reach**, and that path must contain a complete
SeaTunnel installation (`bin/`, `config/`, `lib/`, `connectors/`, etc.). If the
path is empty or missing, DolphinScheduler will fail at job execution time with
a "script not found" or "command not found" error.
+
+## 2. Host Paths vs. Container Paths
+
+This is the most common source of confusion. There are two distinct "points of
view":
+
+- **Host path**: where the SeaTunnel installation actually lives on your
machine or server's disk, e.g. `/home/user/seatunnel` or `./seatunnel`.
+- **Container path**: what DolphinScheduler's container sees internally, e.g.
`/opt/seatunnel`. Whether files exist at that path depends entirely on whether
you mounted a host directory into it via `volumes`.
+
+**Key rule**: `SEATUNNEL_HOME` always describes the path as seen **from the
environment DolphinScheduler's process runs in**, not from your local machine.
If DolphinScheduler runs inside a container, `SEATUNNEL_HOME` must be a
container-internal path (e.g. `/opt/seatunnel`), and that container path must
be mounted to the host directory where SeaTunnel is actually installed.
+
+## 3. Deciding the Path When DolphinScheduler Runs on the Host
+
+If DolphinScheduler is not containerized and runs directly as a process on a
physical or virtual machine:
+
+- `SEATUNNEL_HOME` should be set to the real host-side installation path, e.g.
`/opt/module/seatunnel`.
+- There is no volume-mounting concern here — DolphinScheduler's process and
the SeaTunnel installation share the same filesystem, so the path is exactly
what it appears to be.
+
+## 4. Deciding the Path When DolphinScheduler Also Runs in Docker or Kubernetes
+
+This is the case most people hit with Docker Compose. Two things are required:
+
+1. Mount the host's SeaTunnel installation directory into a path inside the
DolphinScheduler container via `volumes`.
+2. Set `SEATUNNEL_HOME` to the **mounted container-side path**, not the host
path.
+
+The same logic applies in Kubernetes: "mounting" becomes `volumeMounts`
combined with a `PersistentVolume` (or `hostPath`), but the principle is
identical — `SEATUNNEL_HOME` should be set to the path as seen inside the Pod.
+
+## 5. Minimal Working Example (Using the Common Container Path
`/opt/seatunnel`)
+
+```yaml
+version: '3.8'
+
+services:
+ dolphinscheduler:
+ image: apache/dolphinscheduler-standalone-server:3.2.1
+ container_name: dolphinscheduler
+ hostname: dolphinscheduler
+ ports:
+ - "12345:12345"
+ environment:
+ - SEATUNNEL_HOME=/opt/seatunnel
+ volumes:
+ # host ./seatunnel -> container /opt/seatunnel
+ # the host directory must contain a full SeaTunnel installation
beforehand
+ - ./seatunnel:/opt/seatunnel:ro
+ - ./dolphinscheduler/logs:/opt/dolphinscheduler/logs
+ networks:
+ - ds-network
+
+networks:
+ ds-network:
+ driver: bridge
+```
+
+Prepare the host-side SeaTunnel installation directory (the official
DolphinScheduler image does not bundle SeaTunnel itself — it must be downloaded
and extracted manually):
+
+```bash
+export version="3.0.0"
+mkdir -p seatunnel dolphinscheduler/logs
+wget
"https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz";
+tar -zxvf "apache-seatunnel-${version}-bin.tar.gz" -C seatunnel
--strip-components=1
+```
+:::caution Warning
+Since 2.2.0-beta, connector plugins are no longer bundled by default. You must
install them **before** mounting the directory read-only and starting the
container, otherwise every submitted job will fail with a missing-connector
error.
+:::
+
+```bash
+sh seatunnel/bin/install-plugin.sh 3.0.0
+```
+
+After this, `./seatunnel` should contain `bin/`, `config/`, `lib/`, and
similar subdirectories, and the same content will be visible inside the
container at `/opt/seatunnel` once it starts.
+
+## 6. Required Volume Mounts and Network Assumptions
+
+- **Required mount**: the host's SeaTunnel installation directory → the
container path referenced by `SEATUNNEL_HOME`. A read-only mount (`:ro`) is
sufficient, since DolphinScheduler only needs to execute scripts, not write to
the installation directory.
+- **Recommended mounts**: DolphinScheduler's log directory and any directory
holding job configuration files, so logs and configs remain accessible outside
the container for debugging and version control.
+- **Network assumption**: if SeaTunnel jobs need to reach data sources
(databases, message queues, etc.) that are also deployed via Docker Compose,
they must be on the same custom network as DolphinScheduler (e.g. `ds-network`
above); otherwise containers cannot resolve each other by service name and
you'll need to fall back to host IPs or additional network configuration.
+- If SeaTunnel jobs need to reach services on the host machine that are
outside the Docker network, be aware of Docker's default network isolation —
you may need `host.docker.internal` (on Docker Desktop) or explicit
host-network configuration.
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 6b03c8ce66..20b6f38a1a 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -386,7 +386,8 @@ const sidebars = {
"tools/overview",
"tools/seatunnel-skill",
"tools/seatunnel-mcp",
- "tools/x2seatunnel"
+ "tools/x2seatunnel",
+ "tools/dolphinscheduler-integration"
]
},
{
diff --git a/docs/zh/tools/dolphinscheduler-integration.md
b/docs/zh/tools/dolphinscheduler-integration.md
new file mode 100644
index 0000000000..4c3cb58a7d
--- /dev/null
+++ b/docs/zh/tools/dolphinscheduler-integration.md
@@ -0,0 +1,90 @@
+# 使用 Docker Compose 将 SeaTunnel 与 DolphinScheduler 集成
+
+当 SeaTunnel 通过 Docker Compose 部署时，一个常见的问题是：DolphinScheduler 应该配置哪个
`SEATUNNEL_HOME` 路径才能正确提交作业？本文档说明这个问题背后的原理，并给出一个可直接使用的最小示例。
+
+## 1. `SEATUNNEL_HOME` 对调度器集成意味着什么
+
+DolphinScheduler 并不是通过网络接口远程调用 SeaTunnel，而是在自己所在的机器（或容器）上，直接执行 SeaTunnel
提供的启动脚本，例如：
+
+```
+${SEATUNNEL_HOME}/bin/seatunnel.sh --config <job_config_path>
+```
+
+因此 `SEATUNNEL_HOME` 必须指向 **DolphinScheduler 进程实际能够访问到的一个路径**，这个路径下必须包含完整的
SeaTunnel 安装目录（`bin/`、`config/`、`lib/`、`connectors/`
等）。如果这个路径下是空的或者不存在，DolphinScheduler 执行任务时会直接报"找不到脚本"或"命令不存在"的错误。
+
+## 2. 宿主机路径 与 容器内路径 的区别
+
+这是最容易搞混的地方，需要分清两个"视角"：
+
+- **宿主机路径（Host Path）**：SeaTunnel 安装包实际存放在你电脑或服务器磁盘上的位置，例如
`/home/user/seatunnel` 或 `./seatunnel`。
+- **容器内路径（Container Path）**：DolphinScheduler 容器内部看到的路径，例如
`/opt/seatunnel`。这个路径是否存在文件，取决于你有没有通过 `volumes` 把宿主机目录挂载进去。
+
+**关键点**：`SEATUNNEL_HOME` 这个环境变量，配置的永远是 **DolphinScheduler
进程自己所在环境看到的路径**，而不是你电脑上的路径。如果 DolphinScheduler 跑在容器里，`SEATUNNEL_HOME`
就必须写容器内路径（如 `/opt/seatunnel`），并确保这个容器路径通过 volume 挂载对应到了宿主机上真正装了 SeaTunnel 的目录。
+
+## 3. DolphinScheduler 运行在宿主机上时，如何确定路径
+
+如果你没有用容器运行 DolphinScheduler，而是直接在物理机 / 虚拟机上以进程方式运行它，那么：
+
+- `SEATUNNEL_HOME` 直接填宿主机上 SeaTunnel 的真实安装路径，例如 `/opt/module/seatunnel`。
+- 不涉及任何 volume 挂载问题，DolphinScheduler 进程和 SeaTunnel 安装包在同一套文件系统里，路径所见即所得。
+
+## 4. DolphinScheduler 也运行在 Docker（或 Kubernetes）中时，如何确定路径
+
+这是 Docker Compose 场景下最常遇到的情况。此时必须做两件事：
+
+1. 把宿主机上的 SeaTunnel 安装目录，通过 `volumes` 挂载到 DolphinScheduler 容器内的某个路径。
+2. 把 `SEATUNNEL_HOME` 设置为**挂载后的容器内路径**，而不是宿主机路径。
+
+在 Kubernetes 环境下同理，只是"挂载"变成了 `volumeMounts` + `PersistentVolume`（或
`hostPath`），思路完全一致：Pod 内部看到的路径才是 `SEATUNNEL_HOME` 应该填的值。
+
+## 5. 最小可用示例（使用常见容器路径 `/opt/seatunnel`）
+
+```yaml
+version: '3.8'
+
+services:
+ dolphinscheduler:
+ image: apache/dolphinscheduler-standalone-server:3.2.1
+ container_name: dolphinscheduler
+ hostname: dolphinscheduler
+ ports:
+ - "12345:12345"
+ environment:
+ - SEATUNNEL_HOME=/opt/seatunnel
+ volumes:
+ # 宿主机 ./seatunnel 目录 -> 容器内 /opt/seatunnel
+ # 宿主机这个目录下需要预先放好完整的 SeaTunnel 安装包内容
+ - ./seatunnel:/opt/seatunnel:ro
+ - ./dolphinscheduler/logs:/opt/dolphinscheduler/logs
+ networks:
+ - ds-network
+
+networks:
+ ds-network:
+ driver: bridge
+```
+
+准备宿主机上的 SeaTunnel 安装目录（Docker 官方镜像不自带 SeaTunnel 程序本体，需要手动下载解压）：
+
+```bash
+export version="3.0.0"
+mkdir -p seatunnel dolphinscheduler/logs
+wget
"https://archive.apache.org/dist/seatunnel/${version}/apache-seatunnel-${version}-bin.tar.gz";
+tar -zxvf "apache-seatunnel-${version}-bin.tar.gz" -C seatunnel
--strip-components=1
+```
+:::caution 警告
+从 2.2.0-beta
版本开始，连接器插件默认不再随安装包一起打包。你必须在把目录以只读方式挂载并启动容器**之前**安装好这些插件，否则所有提交的作业都会因为缺少连接器而失败。
+:::
+
+```bash
+sh seatunnel/bin/install-plugin.sh 3.0.0
+```
+
+完成后，`./seatunnel` 目录下应包含 `bin/`、`config/`、`lib/` 等子目录，容器启动后即可在
`/opt/seatunnel` 下看到同样的内容。
+
+## 6. 所需的 Volume 挂载与网络假设
+
+- **必须挂载**：宿主机的 SeaTunnel 安装目录 → 容器内 `SEATUNNEL_HOME` 指向的路径（只读挂载 `:ro` 即可，因为
DolphinScheduler 只需要执行脚本，不需要写入安装目录）。
+- **建议挂载**：DolphinScheduler 的日志目录、以及存放作业配置文件（job
config）的目录，方便在容器外查看任务配置和日志，也便于版本管理和排查问题。
+- **网络假设**：如果 SeaTunnel 需要连接的数据源（数据库、消息队列等）也用 Docker Compose 部署，需确保它们和
DolphinScheduler 处于同一个自定义网络（如上例中的 `ds-network`），否则容器间无法通过服务名互相访问，必须改用宿主机 IP
或额外的网络配置。
+- 如果 SeaTunnel 任务需要访问宿主机上的其他服务（不在 Docker 网络内），需要注意 Docker 默认的网络隔离，可能需要使用
`host.docker.internal`（Docker Desktop 环境）或显式配置宿主机网络访问。
