This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 5e2a864d29c [feat](i18n) Add manual-only Korean translation workflow
(#4358)
5e2a864d29c is described below
commit 5e2a864d29ca9c932c4e14f08a2f800db37493eb
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sun Oct 18 18:33:28 2026 +0100
[feat](i18n) Add manual-only Korean translation workflow (#4358)
Add a manual Korean translation workflow that defaults to a read-only
dry run. It fixes `I18N_LOCALE=ko`, keeps Korean artifacts and temporary
paths separate, and shares the existing `japanese-translation-ja`
whole-run lock with Japanese translation.
The workflow reuses the existing scan, Codex generation, auth refresh,
and PR publisher. Validation requires an exact source file and limits
generation to one page. It verifies `translate-ko` before loading model
auth and retains normal PR admission checks and labeling. No schedule is
added.
Validation: 210 Node 20 tests ran on the integrated T04/T06 base: 209
passed, none failed, and one existing Apache integration test was
skipped. Three workflow contract tests cover manual defaults,
locale/permissions/concurrency, exact-commit artifact handoff, and
label/auth ordering. No real model call has been made; dry-run and
bounded live validation follow after merge. This
workflow/test/documentation-only change skips the website build under
the existing scope rules.
Co-authored-by: morningman <[email protected]>
---
.github/workflows/build-check.yml | 1 +
.github/workflows/korean-translation.yml | 226 ++++++++++++++++++++++++
scripts/i18n/__tests__/korean-workflow.test.cjs | 69 ++++++++
translations/README.md | 20 +++
4 files changed, 316 insertions(+)
diff --git a/.github/workflows/build-check.yml
b/.github/workflows/build-check.yml
index 4144efc41a3..dc6f3208719 100644
--- a/.github/workflows/build-check.yml
+++ b/.github/workflows/build-check.yml
@@ -114,6 +114,7 @@ jobs:
node --test scripts/i18n/__tests__/planner.test.cjs
node --test
scripts/i18n/__tests__/translation-engine.test.cjs
node --test scripts/i18n/__tests__/korean-toolchain.test.cjs
+ node --test scripts/i18n/__tests__/korean-workflow.test.cjs
node --test scripts/i18n/__tests__/workflow.test.cjs
node --test scripts/i18n/__tests__/codex-adapter.test.cjs
node --test
scripts/i18n/__tests__/translate-document.test.cjs
diff --git a/.github/workflows/korean-translation.yml
b/.github/workflows/korean-translation.yml
new file mode 100644
index 00000000000..1c07a2e0c55
--- /dev/null
+++ b/.github/workflows/korean-translation.yml
@@ -0,0 +1,226 @@
+name: Korean translation discovery
+
+on:
+ workflow_dispatch:
+ inputs:
+ mode:
+ description: Read-only dry run, one-page live validation, or batch
generation
+ type: choice
+ options: [dry-run, validate, generate]
+ default: dry-run
+ required: true
+ plugin:
+ description: Optional plugin filter (default or community)
+ type: string
+ version:
+ description: Optional version filter (current or 4.x)
+ type: string
+ path:
+ description: Optional repository-relative source path prefix
+ type: string
+
+permissions:
+ contents: read
+ pull-requests: read
+
+concurrency:
+ # One entire run across scheduled/manual triggers, locales and modes.
+ # Keep this stable so an in-flight run also holds the same lock.
+ group: japanese-translation-ja
+ cancel-in-progress: false
+ queue: max
+
+jobs:
+ discover:
+ if: github.ref == 'refs/heads/master'
+ runs-on: ubuntu-latest
+ timeout-minutes: 20
+ env:
+ I18N_LOCALE: ko
+ outputs:
+ can_generate: ${{ steps.scan.outputs.can_generate }}
+ source_commit: ${{ steps.scan.outputs.source_commit }}
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ ref: master
+ fetch-depth: 0
+ persist-credentials: false
+ - uses: actions/setup-node@v4
+ with:
+ node-version: 20
+ cache: yarn
+ - run: yarn --frozen-lockfile
+ - name: Reconcile every open master PR with managed Korean files
+ env:
+ GH_TOKEN: ${{ github.token }}
+ GITHUB_REPOSITORY: ${{ github.repository }}
+ run: node scripts/i18n/workflow-prs.cjs
"$RUNNER_TEMP/korean-translation"
+ - name: Plan all current and historical pending documents
+ id: scan
+ env:
+ I18N_MODE: ${{ inputs.mode || 'dry-run' }}
+ I18N_PLUGIN: ${{ inputs.plugin }}
+ I18N_VERSION: ${{ inputs.version }}
+ I18N_PATH: ${{ inputs.path }}
+ I18N_LIMIT: '20'
+ run: node scripts/i18n/workflow-scan.cjs
"$RUNNER_TEMP/korean-translation"
+ - uses: actions/upload-artifact@v4
+ if: always()
+ with:
+ name: korean-translation-plan-${{ github.run_id }}
+ path: ${{ runner.temp }}/korean-translation/
+ if-no-files-found: warn
+ retention-days: 30
+
+ generate:
+ needs: discover
+ env:
+ I18N_LOCALE: ko
+ if: needs.discover.outputs.can_generate == 'true'
+ runs-on: ubuntu-latest
+ timeout-minutes: 120
+ steps:
+ - uses: actions/checkout@v4
+ with:
+ ref: ${{ needs.discover.outputs.source_commit }}
+ fetch-depth: 0
+ persist-credentials: false
+ - uses: actions/setup-node@v4
+ with:
+ node-version: 20
+ cache: yarn
+ - uses: actions/setup-python@v4
+ with:
+ python-version: '3.9'
+ - run: yarn --frozen-lockfile
+ - uses: actions/download-artifact@v4
+ with:
+ name: korean-translation-plan-${{ github.run_id }}
+ path: ${{ runner.temp }}/korean-translation
+ - name: Check external configuration
+ id: config
+ env:
+ OSS_AK: ${{ secrets.OSS_AK }}
+ OSS_SK: ${{ secrets.OSS_SK }}
+ APP_ID: ${{ secrets.I18N_GITHUB_APP_ID }}
+ APP_PRIVATE_KEY: ${{ secrets.I18N_GITHUB_APP_PRIVATE_KEY }}
+ I18N_GITHUB_PAT: ${{ secrets.I18N_GITHUB_PAT }}
+ I18N_MODE: ${{ inputs.mode || 'dry-run' }}
+ run: |
+ if [[ -n "$APP_ID" && -n "$APP_PRIVATE_KEY" ]]; then
+ echo 'auth_kind=app' >> "$GITHUB_OUTPUT"
+ elif [[ -n "$I18N_GITHUB_PAT" ]]; then
+ echo 'auth_kind=pat' >> "$GITHUB_OUTPUT"
+ else
+ echo 'auth_kind=none' >> "$GITHUB_OUTPUT"
+ fi
+ if [[ -n "$OSS_AK" && -n "$OSS_SK" && ( -n "$APP_ID" && -n
"$APP_PRIVATE_KEY" || -n "$I18N_GITHUB_PAT" || "$I18N_MODE" == 'validate' ) ]];
then
+ echo 'ready=true' >> "$GITHUB_OUTPUT"
+ else
+ echo 'ready=false' >> "$GITHUB_OUTPUT"
+ echo 'OSS Codex auth or a GitHub write credential is missing. No
model call or PR write was attempted.' >> "$GITHUB_STEP_SUMMARY"
+ fi
+ - name: Verify personal token identity before model calls
+ if: steps.config.outputs.auth_kind == 'pat'
+ env:
+ GH_TOKEN: ${{ secrets.I18N_GITHUB_PAT }}
+ run: gh api user --jq '.login' > /dev/null
+ - name: Reconcile again immediately before model calls
+ if: steps.config.outputs.ready == 'true'
+ env:
+ GH_TOKEN: ${{ github.token }}
+ GITHUB_REPOSITORY: ${{ github.repository }}
+ run: node scripts/i18n/workflow-prs.cjs
"$RUNNER_TEMP/korean-translation"
+ - name: Replan exact source and open PRs
+ if: steps.config.outputs.ready == 'true'
+ id: fresh
+ env:
+ I18N_MODE: ${{ inputs.mode || 'dry-run' }}
+ I18N_PLUGIN: ${{ inputs.plugin }}
+ I18N_VERSION: ${{ inputs.version }}
+ I18N_PATH: ${{ inputs.path }}
+ I18N_LIMIT: '20'
+ run: node scripts/i18n/workflow-scan.cjs
"$RUNNER_TEMP/korean-translation"
+ - name: Verify the Korean PR label before model calls
+ if: steps.fresh.outputs.can_generate == 'true'
+ env:
+ GH_TOKEN: ${{ github.token }}
+ run: test "$(gh api repos/apache/doris-website/labels/translate-ko
--jq .name)" = translate-ko
+ - name: Install ossutil for the pinned Codex package and auth pool
+ if: steps.fresh.outputs.can_generate == 'true'
+ run: |
+ temp_dir="$(mktemp -d)"
+ trap 'rm -rf "$temp_dir"' EXIT
+ curl -fsSL -o "$temp_dir/ossutil.zip"
https://gosspublic.alicdn.com/ossutil/1.7.19/ossutil-v1.7.19-linux-amd64.zip
+ unzip -q "$temp_dir/ossutil.zip" -d "$temp_dir"
+ sudo install -m 0755 "$temp_dir/ossutil-v1.7.19-linux-amd64/ossutil"
/usr/local/bin/ossutil
+ - name: Install the verified Codex runtime used by Doris review
+ if: steps.fresh.outputs.can_generate == 'true'
+ env:
+ OSS_AK: ${{ secrets.OSS_AK }}
+ OSS_SK: ${{ secrets.OSS_SK }}
+ run: |
+ archive="$RUNNER_TEMP/codex-exec-goal.tar.gz"
+ package="$RUNNER_TEMP/codex-exec-goal"
+ ossutil -i "$OSS_AK" -k "$OSS_SK" -e oss-cn-hongkong.aliyuncs.com \
+ cp -f
oss://doris-community-ci/codex/releases/codex-exec-goal-0.156.1-a310d33f7ebb-x86_64-unknown-linux-gnu.tar.gz
"$archive"
+ echo
"2ee3073479bc8c94436684b55a13d7a3f299e768d04436d90e0427814a3954bc $archive" |
sha256sum --check
+ mkdir -p "$package"
+ tar -xzf "$archive" -C "$package"
+ (cd "$package" && sha256sum --check SHA256SUMS)
+ test "$(jq -r .version "$package/codex-package.json")" = 0.156.1
+ "$package/bin/codex" exec --help | grep -q -- '--output-schema'
+ "$package/bin/codex" exec --help | grep -q -- '--disable'
+ "$package/bin/codex" exec --help | grep -q -- '--ignore-user-config'
+ "$package/bin/codex" exec --help | grep -q -- '--ephemeral'
+ echo "CODEX_BIN=$package/bin/codex" >> "$GITHUB_ENV"
+ - name: Load an eligible Codex auth snapshot
+ if: steps.fresh.outputs.can_generate == 'true'
+ id: auth
+ env:
+ OSS_AK: ${{ secrets.OSS_AK }}
+ OSS_SK: ${{ secrets.OSS_SK }}
+ run: node scripts/i18n/codex-auth.cjs load
+ - name: Generate a bounded, validated batch
+ if: steps.auth.outcome == 'success'
+ id: translation
+ env:
+ I18N_CODEX_MODEL: ${{ vars.I18N_CODEX_MODEL || 'gpt-6-sol' }}
+ I18N_MODE: ${{ inputs.mode || 'dry-run' }}
+ I18N_PATH: ${{ inputs.path }}
+ I18N_MAX_PAGES: ${{ inputs.mode == 'validate' && '1' || '10' }}
+ GITHUB_REPOSITORY: ${{ github.repository }}
+ run: node scripts/i18n/workflow-generate.cjs
"$RUNNER_TEMP/korean-translation"
+ - uses: actions/create-github-app-token@v2
+ if: steps.translation.outputs.created == 'true' &&
steps.config.outputs.auth_kind == 'app'
+ id: app
+ with:
+ app-id: ${{ secrets.I18N_GITHUB_APP_ID }}
+ private-key: ${{ secrets.I18N_GITHUB_APP_PRIVATE_KEY }}
+ owner: apache
+ repositories: doris-website
+ permission-contents: write
+ permission-pull-requests: write
+ - name: Run shared PR admission checks and open reviewable PR
+ if: steps.translation.outputs.created == 'true' &&
steps.config.outputs.auth_kind != 'none'
+ env:
+ GH_TOKEN: ${{ steps.app.outputs.token || secrets.I18N_GITHUB_PAT }}
+ I18N_PR_AUTH_KIND: ${{ steps.config.outputs.auth_kind }}
+ run: node scripts/i18n/workflow-publish.cjs
"$RUNNER_TEMP/korean-translation"
+ - name: Report staged validation without GitHub write credentials
+ if: steps.translation.outputs.created == 'true' &&
steps.config.outputs.auth_kind == 'none'
+ run: echo 'One-page Codex candidate is in the generation artifact. A
GitHub App or personal token is required for a translation PR.' >>
"$GITHUB_STEP_SUMMARY"
+ - name: Sync refreshed Codex auth without overwriting a newer snapshot
+ if: always() && steps.auth.outcome == 'success'
+ env:
+ OSS_AK: ${{ secrets.OSS_AK }}
+ OSS_SK: ${{ secrets.OSS_SK }}
+ run: node scripts/i18n/codex-auth.cjs sync
+ - uses: actions/upload-artifact@v4
+ if: always()
+ with:
+ name: korean-translation-generation-${{ github.run_id }}
+ path: ${{ runner.temp }}/korean-translation/
+ if-no-files-found: warn
+ retention-days: 30
diff --git a/scripts/i18n/__tests__/korean-workflow.test.cjs
b/scripts/i18n/__tests__/korean-workflow.test.cjs
new file mode 100644
index 00000000000..640f3956b2c
--- /dev/null
+++ b/scripts/i18n/__tests__/korean-workflow.test.cjs
@@ -0,0 +1,69 @@
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const yaml = require('js-yaml');
+const workflows = path.resolve(__dirname, '../../../.github/workflows');
+const load = name => yaml.load(fs.readFileSync(path.join(workflows, name),
'utf8'));
+const korean = load('korean-translation.yml');
+const japanese = load('japanese-translation.yml');
+
+test('Korean generation requires a manual choice and shares the existing
whole-run lock', () => {
+ assert.deepEqual(Object.keys(korean.on), ['workflow_dispatch']);
+ assert.equal(korean.on.workflow_dispatch.inputs.mode.default, 'dry-run');
+ assert.deepEqual(korean.on.workflow_dispatch.inputs.mode.options,
['dry-run', 'validate', 'generate']);
+ assert.equal(korean.on.workflow_dispatch.inputs.locale, undefined);
+ assert.deepEqual(korean.permissions, {'contents':'read',
'pull-requests':'read'});
+ assert.deepEqual(korean.concurrency, japanese.concurrency);
+ assert.equal(korean.concurrency.group, 'japanese-translation-ja');
+ assert.equal(korean.jobs.discover.if, "github.ref == 'refs/heads/master'");
+ assert.equal(korean.jobs.generate.needs, 'discover');
+ assert.equal(korean.jobs.generate.if, "needs.discover.outputs.can_generate
== 'true'");
+ for (const job of Object.values(korean.jobs)) {
+ assert.equal(job.env.I18N_LOCALE, 'ko');
+ for (const step of job.steps) {
+ if (step.env?.I18N_MODE) assert.equal(step.env.I18N_MODE, "${{
inputs.mode || 'dry-run' }}");
+ if (step.uses?.startsWith('actions/checkout@')) {
+ assert.equal(step.with['fetch-depth'], 0);
+ assert.equal(step.with['persist-credentials'], false);
+ }
+ }
+ }
+});
+
+test('Korean discovery and generation exchange Korean artifacts at the exact
scanned commit', () => {
+ const discover = korean.jobs.discover.steps;
+ const generate = korean.jobs.generate.steps;
+ assert.equal(generate.find(step =>
step.uses?.startsWith('actions/checkout@')).with.ref,
+ '${{ needs.discover.outputs.source_commit }}');
+ const upload = discover.find(step =>
step.uses?.startsWith('actions/upload-artifact@'));
+ const download = generate.find(step =>
step.uses?.startsWith('actions/download-artifact@'));
+ assert.equal(upload.with.name, 'korean-translation-plan-${{ github.run_id
}}');
+ assert.equal(download.with.name, upload.with.name);
+ assert.equal(download.with.path, '${{ runner.temp }}/korean-translation');
+ const scripts = [...discover, ...generate].filter(step => /node
scripts\/i18n\/workflow-(prs|scan|generate|publish)/.test(step.run ?? ''));
+ assert.equal(scripts.length, 6);
+ assert(scripts.every(step =>
step.run.includes('"$RUNNER_TEMP/korean-translation"')));
+ const execute = generate.find(step => step.id === 'translation');
+ assert.equal(execute.env.I18N_MAX_PAGES, "${{ inputs.mode == 'validate' &&
'1' || '10' }}");
+ assert.equal(execute.env.I18N_PATH, '${{ inputs.path }}');
+ assert.equal(execute.env.I18N_CODEX_MODEL,
japanese.jobs.generate.steps.find(step => step.id ===
'translation').env.I18N_CODEX_MODEL);
+});
+
+test('Korean publication checks the label before auth loading and preserves
the verified runtime and refresh path', () => {
+ const steps = korean.jobs.generate.steps;
+ const label = steps.findIndex(step => step.name === 'Verify the Korean PR
label before model calls');
+ const auth = steps.findIndex(step => step.id === 'auth');
+ assert(label >= 0 && label < auth);
+ assert.equal(steps[label].if, "steps.fresh.outputs.can_generate ==
'true'");
+ assert.equal(steps[label].env.GH_TOKEN, '${{ github.token }}');
+ assert(steps[label].run.includes('labels/translate-ko'));
+ const runtime = steps.find(step => step.name === 'Install the verified
Codex runtime used by Doris review');
+ assert.deepEqual(runtime, japanese.jobs.generate.steps.find(step =>
step.name === runtime.name));
+ const sync = steps.find(step => step.name === 'Sync refreshed Codex auth
without overwriting a newer snapshot');
+ assert.equal(sync.if, "always() && steps.auth.outcome == 'success'");
+ assert.equal(sync.run, 'node scripts/i18n/codex-auth.cjs sync');
+ const publish = steps.find(step =>
step.run?.includes('workflow-publish.cjs'));
+ assert.equal(publish.if, "steps.translation.outputs.created == 'true' &&
steps.config.outputs.auth_kind != 'none'");
+ assert.equal(publish.env.GH_TOKEN, '${{ steps.app.outputs.token ||
secrets.I18N_GITHUB_PAT }}');
+});
diff --git a/translations/README.md b/translations/README.md
index 1db4180b3d1..c27d3018da7 100644
--- a/translations/README.md
+++ b/translations/README.md
@@ -103,6 +103,26 @@ existing source/technical hash algorithms unchanged.
Offline Korean fixtures
exercise these boundaries without real credentials or model calls; actual
single-page generation is a separate validation step.
+The `Korean translation discovery` workflow is manual-only and defaults to a
+read-only dry run. It fixes the locale to `ko`, uses separate Korean artifacts,
+and shares the existing `japanese-translation-ja` lock with Japanese runs.
+After the workflow is merged, run discovery on master:
+
+```bash
+gh workflow run korean-translation.yml --repo apache/doris-website --ref
master -f mode=dry-run
+```
+
+For a bounded real validation, select one eligible English file from the plan
+and provide its exact repository-relative `.md` or `.mdx` path with
+`-f mode=validate -f path=<exact-source-file>`. Validation sets the page limit
+to one and rechecks active PRs before calling Codex. Generation checks the
+`translate-ko` label before loading auth, then uses the existing verified
+runtime and auth pool. A write credential enables a normal labeled PR after
+shared admission checks; otherwise a successful one-page validation remains
+an artifact and does not establish the PR publication path. Inspect generation
+warnings and normal PR checks before merging. There is no Korean generation
+schedule in this workflow.
+
## Search, navigation and four-language validation
`check-configuration.cjs --all` validates each managed locale's scope and
glossary
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
