This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new eda6edcb0741 [improve](ci) Avoid full builds for landing translation
metadata (#4520)
eda6edcb0741 is described below
commit eda6edcb07417add48d942e6d6cf5d6627f7727e
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Mon Oct 26 07:47:49 2026 +0000
[improve](ci) Avoid full builds for landing translation metadata (#4520)
PR #4516 spent 125m54s building the complete five-language site, then
failed after another 17m11s of static verification. The actual failure
was an obsolete browser assertion: `/ja/download/` was required to have
an English body even though the PR publishes a Japanese translation.
Two known inputs, `translations/COMMUNITY_ENTRIES.md` and
`translations/community-reports.json`, were classified as unknown and
escalated the PR to full compilation. Treat top-level translation
maintenance notes as prose and the report registry as shared page
content, so landing translation PRs use the existing representative
build. Ordinary pages remain fully rendered; document samples span
configured languages and active versions. Dependencies, versions,
unknown inputs and explicit `ci/full` requests retain full validation.
Run sampling, native content and publishing fixtures only for their
respective subsystems. Skip unchanged translation unit tests for
document/dictionary/state edits and the full docs governance scan for
landing-only edits. Keep browser tools available for independently
selected fixtures. Browser language/fallback expectations now follow the
native manifest; the content fixture exercises both English fallback and
translated bodies.
Validation:
- Replayed all 30 changed paths from #4516: `full` becomes `sample`;
publishing fixture and docs governance are omitted.
- 286 i18n tests passed, one existing skip; 22 workflow/security tests
passed; actionlint and `git diff --check` passed.
- Reproduced the original `ja !== en` failure against #4516's existing
local five-language landing artifact. The updated verifier passed
language navigation, disabled storage/JavaScript and 1440/390 layouts
across eight themes.
- Fresh native fixture builds passed in five languages and with Russian
disabled (nine locale builds, 162 checked pages). The updated CLI
verifier passed both fallback and translated bodies against the fresh
artifact. The real #4516 browser reproduction reused its existing local
landing artifact; no new full-site compilation or production deployment
is claimed.
Self-review against AGENTS.md:
- Goal: fixes unnecessary full-build selection and the stale language
assertion; the exact failing PR's file set now selects representative
compilation.
- Scope: six CI, test and maintenance-document files; existing build and
sampling implementations are reused.
- Information architecture: document trees, versions, locale content and
sidebars are unchanged.
- Links/navigation: no public paths, anchors, slugs or redirects change.
- Frontend/build conventions: no UI changes; production builds and
full-validation overrides remain available.
- Validation/security: read-only PR permissions and credential-free
checkout remain; checks above cover policy, workflow wiring and browser
behavior. GitHub CI will validate fresh fixture builds.
- Other issues: samples retain their publication prohibition and
explicit coverage limits. No additional issue found.
---------
Co-authored-by: morningman <[email protected]>
---
.github/workflows/build-check.yml | 6 ++---
scripts/i18n/CI_VALIDATION.md | 14 +++++++++-
scripts/i18n/__tests__/ci-sampling.test.cjs | 40 +++++++++++++++++++++++++++++
scripts/i18n/__tests__/ci-scope.test.cjs | 5 ++++
scripts/i18n/ci-plan.cjs | 24 ++++++++++++-----
scripts/i18n/verify-global-language.cjs | 23 ++++++++++++-----
6 files changed, 95 insertions(+), 17 deletions(-)
diff --git a/.github/workflows/build-check.yml
b/.github/workflows/build-check.yml
index 1445da03a333..4fedd9c21bad 100644
--- a/.github/workflows/build-check.yml
+++ b/.github/workflows/build-check.yml
@@ -120,7 +120,7 @@ jobs:
- name: Translation unit tests
env:
NODE_OPTIONS: --max-old-space-size=8192
- if: steps.detect.outputs.validation_mode != 'skip'
+ if: steps.detect.outputs.translation_tests == 'true'
run: node --test scripts/i18n/__tests__/*.test.cjs
- name: Publish artifact optimization tests
@@ -160,7 +160,7 @@ jobs:
done
- name: Install browser verification tools
- if: steps.detect.outputs.validation_mode == 'full' ||
steps.detect.outputs.validation_mode == 'sample' ||
steps.detect.outputs.publish_fixture == 'true'
+ if: steps.detect.outputs.validation_mode == 'full' ||
steps.detect.outputs.validation_mode == 'sample' ||
steps.detect.outputs.publish_fixture == 'true' ||
steps.detect.outputs.content_fixture == 'true' ||
steps.detect.outputs.sampling_fixture == 'true'
run: |
DORIS_BROWSER_TOOLS="$RUNNER_TEMP/doris-browser-tools"
npm install --prefix "$DORIS_BROWSER_TOOLS" --no-save
--no-package-lock [email protected]
@@ -263,7 +263,7 @@ jobs:
retention-days: 14
- name: Docs governance report
- if: always() && steps.install.outcome == 'success' &&
steps.detect.outputs.validation_mode != 'fixtures'
+ if: always() && steps.install.outcome == 'success' &&
steps.detect.outputs.docs_report == 'true'
continue-on-error: true
run: |
yarn docs:lint:changed --output
website-quality-governance/generated/docs-governance-report.json
diff --git a/scripts/i18n/CI_VALIDATION.md b/scripts/i18n/CI_VALIDATION.md
index 97671d2bf1f5..d24a937c0eb6 100644
--- a/scripts/i18n/CI_VALIDATION.md
+++ b/scripts/i18n/CI_VALIDATION.md
@@ -9,7 +9,7 @@ committed lockfile after source planning. Source snapshot
checks remain enabled.
| Change | Validation |
| --- | --- |
| README, planning or review notes only | Configuration, move/link and
workflow security checks; no dependency installation or site build |
-| CI and tooling | Unit tests, five-language native compilation fixture and
publishing/Apache/Chromium fixture |
+| CI and tooling | Applicable unit tests and fixtures for the changed
subsystem; no real-site build |
| Shared UI, shared UI registry, routing, SEO, localization or sidebars |
Representative real documents in five languages and every active version;
native content fixtures for content/locale implementations, publishing fixtures
for delivery/host implementations |
| Documentation or assets | Changed documents and import dependants, plus
representative real documents; English source changes select all five
languages, localized changes select English and the affected locales |
| Dependencies, active versions or unclassified inputs | Full five-language
publication build and verification |
@@ -26,6 +26,18 @@ configured language and active version, along with the
native content fixture.
Ordinary pages remain in the real build. This classification applies only to
that exact registry path; other unknown inputs still require full validation.
+Top-level `translations/*.md` files are maintenance notes and do not request a
+site build. `translations/community-reports.json` controls ordinary page
content
+and selects representative real builds. Combining these known inputs with a
+landing-page translation no longer upgrades it to a full publication build.
+Other unknown JSON inputs still require full validation.
+
+The sampling adapter, native content and publishing/Apache fixtures run only
+when their own implementation changes. Document, dictionary and
translation-state
+edits retain content admission and real sampled compilation without rerunning
+unchanged translation unit tests. Docs governance runs for document/navigation
+changes and full validation, rather than scanning all docs for a landing UI
edit.
+
## What a sample proves
The CI adapter first loads native Docusaurus metadata for **all documents and
diff --git a/scripts/i18n/__tests__/ci-sampling.test.cjs
b/scripts/i18n/__tests__/ci-sampling.test.cjs
index 288c875663ec..3d482260982d 100644
--- a/scripts/i18n/__tests__/ci-sampling.test.cjs
+++ b/scripts/i18n/__tests__/ci-sampling.test.cjs
@@ -47,6 +47,46 @@ test('shared UI registry selects real samples and content
fixtures while unknown
assert.equal(policy([registry], {forceFull: true}).mode, 'full');
assert.equal(policy([registry, 'translations/shared/unknown.json'],
{forceSample: true}).mode, 'full');
});
+test('community translation metadata does not upgrade a landing-page PR to
full compilation', () => {
+ const files = ['translations/COMMUNITY_ENTRIES.md',
'translations/community-reports.json',
+ 'translations/pages/registry.json', 'translations/pages/state.json',
+ 'i18n/ja/code.json',
'src/components/community-report-next/CommunityReportNext.tsx'];
+ const selected = policy(files);
+ assert.equal(selected.mode, 'sample');
+ assert.equal(selected.selection, 'representative');
+ assert.equal(selected.publicationAllowed, false);
+ assert.equal(selected.runPublishFixture, false);
+ assert.equal(selected.runDocsReport, false);
+ assert.equal(policy(['translations/community-reports.json']).mode,
'sample');
+ for (const note of ['COMMUNITY_ENTRIES', 'FIRST_RELEASE', 'SHARED_UI',
'README'])
+ assert.equal(policy([`translations/${note}.md`]).mode, 'skip');
+ assert.equal(policy([...files, 'translations/unknown.json']).mode, 'full');
+ assert.equal(policy(files, {forceFull: true}).mode, 'full');
+});
+test('CI tooling only runs the fixtures for its own subsystem', () => {
+ const language = policy(['scripts/i18n/verify-global-language.cjs']);
+ assert.equal(language.mode, 'fixtures');
+ assert.equal(language.runContentFixture, true);
+ assert.equal(language.runSamplingFixture, false);
+ assert.equal(language.runPublishFixture, false);
+ const sampler = policy(['scripts/i18n/ci-sample.cjs']);
+ assert.equal(sampler.runSamplingFixture, true);
+ assert.equal(sampler.runPublishFixture, false);
+ const publishing = policy(['scripts/publish/apache.cjs']);
+ assert.equal(publishing.runPublishFixture, true);
+ assert.equal(publishing.runSamplingFixture, false);
+ assert.equal(policy(['.github/workflows/build-check.yml']).runDocsReport,
false);
+ const docs = policy(['docs/topic.md']);
+ assert.equal(docs.runTranslationTests, false);
+ assert.equal(docs.runDocsReport, true);
+ const dictionary = policy(['i18n/ja/code.json']);
+ assert.equal(dictionary.runTranslationTests, false);
+ assert.equal(dictionary.runDocsReport, false);
+ assert.equal(policy(['src/utils/localizedRoute.ts']).runTranslationTests,
true);
+ const full = policy([], {forceFull: true});
+ assert.equal(full.runTranslationTests, true);
+ assert.equal(full.runDocsReport, true);
+});
test('a registry-only commit samples all configured languages and active
versions without claiming publication coverage', t => {
const root = directory(t), registry = 'translations/shared/registry.json';
fs.mkdirSync(path.join(root, 'translations/shared'), {recursive: true});
diff --git a/scripts/i18n/__tests__/ci-scope.test.cjs
b/scripts/i18n/__tests__/ci-scope.test.cjs
index fc304216a9fe..e2a95efa5433 100644
--- a/scripts/i18n/__tests__/ci-scope.test.cjs
+++ b/scripts/i18n/__tests__/ci-scope.test.cjs
@@ -58,9 +58,14 @@ test('CI fails early on cheap checks, retains required job
identity and deduplic
assert(index('Publish artifact optimization tests') < index('Translation
gate for changed locales'));
assert(index('Publish artifact optimization tests') < index('Verify native
content manifest fixture'));
assert.match(steps[index('Translation unit tests')].run,
/__tests__\/\*\.test\.cjs/);
+ assert.equal(steps[index('Translation unit tests')].if,
"steps.detect.outputs.translation_tests == 'true'");
assert.match(steps[index('Install dependencies')].run, /\byarn
--frozen-lockfile\b/);
assert.equal(steps.filter(step => /docs:links:changed/.test(step.run ??
'')).length, 0);
assert.match(steps[index('Docs governance report')].run,
/docs:lint:changed/);
+ assert.match(steps[index('Docs governance report')].if,
/steps\.detect\.outputs\.docs_report == 'true'/);
+ const browserTools = steps[index('Install browser verification tools')].if;
+ for (const fixture of ['content_fixture', 'publish_fixture',
'sampling_fixture'])
+ assert(browserTools.includes(`steps.detect.outputs.${fixture} ==
'true'`), fixture);
const verify = steps[index('Verify localized static pages')].run;
assert.match(verify, /\[ "\$LOCALE" = "ru" \]; then
EXTRA_ARGS\+=\(--source-locales/);
assert.match(verify, /\[ "\$LOCALE" = "ko" \] \|\| \[ "\$LOCALE" = "ru"
\]; then EXTRA_ARGS=\(--themes\)/);
diff --git a/scripts/i18n/ci-plan.cjs b/scripts/i18n/ci-plan.cjs
index 941185b418d9..0bf2e69f3a46 100644
--- a/scripts/i18n/ci-plan.cjs
+++ b/scripts/i18n/ci-plan.cjs
@@ -14,10 +14,14 @@ const runtime =
const content =
/^(static\/|docs\/|versioned_docs\/|community\/|course\/|releasenotes\/|blog\/|i18n\/|translations\/(pages|blog|state)\/)/;
const highRisk =
/^(package\.json|yarn\.lock|versions\.json|config\/versions-plugin\.js)$/;
+// Maintenance notes are not site content. The report registry, in contrast,
+// changes ordinary pages, which the real sample build always renders.
+const maintenanceNotes = /^translations\/[^/]+\.md$/;
+const reportRegistry = 'translations/community-reports.json';
function policy(files, {forceFull = false, forceSample = false} = {}) {
const risk = files.filter(file => highRisk.test(file));
const changed = files.some(file => content.test(file));
- const shared = files.some(file => runtime.test(file) ||
mechanism.test(file));
+ const shared = files.some(file => runtime.test(file) ||
mechanism.test(file) || file === reportRegistry);
const checks = validationScope(files);
const ci = files.some(file => /^(scripts\/|\.github\/)/.test(file));
const unknown = files.filter(
@@ -25,7 +29,9 @@ function policy(files, {forceFull = false, forceSample =
false} = {}) {
!runtime.test(file) &&
!mechanism.test(file) &&
!content.test(file) &&
-
!/^((scripts|\.github|plan-doc|review-docs|\.agents)\/|translations\/CONTENT_MANIFEST\.md$|README\.md$|AGENTS\.md$|LICENSE$|NOTICE$)/.test(
+ file !== reportRegistry &&
+ !maintenanceNotes.test(file) &&
+
!/^((scripts|\.github|plan-doc|review-docs|\.agents)\/|README\.md$|AGENTS\.md$|LICENSE$|NOTICE$)/.test(
file,
),
);
@@ -50,11 +56,15 @@ function policy(files, {forceFull = false, forceSample =
false} = {}) {
runContentFixture:
(shared && checks.check_content_fixture) ||
files.some(file =>
-
/verify-content-fixture|__tests__\/fixtures\/content-manifest/.test(file),
+
/verify-(content-fixture|global-language)|__tests__\/fixtures\/content-manifest/.test(file),
),
- runSamplingFixture: mode === 'fixtures' || files.some(file =>
-
/^scripts\/(build-locales?\.cjs$|i18n\/(ci-sample|ci-blog-sample|build-cache)\.cjs$)/.test(file)),
- runPublishFixture: mode === 'fixtures' || checks.check_publish_fixture
|| mode === 'full',
+ runSamplingFixture: files.some(file =>
+
/^scripts\/(build-locales?\.cjs$|i18n\/(ci-sample|ci-blog-sample|build-cache|verify-ci-sampl(?:e|ing-fixture))\.cjs$)/.test(file)),
+ runPublishFixture: checks.check_publish_fixture || mode === 'full',
+ runTranslationTests: mode === 'full' || files.some(file =>
+ runtime.test(file) || mechanism.test(file) ||
/^scripts\/(i18n\/|build-locales?\.cjs$)/.test(file)
+ || file === '.github/workflows/build-check.yml'),
+ runDocsReport: mode === 'full' || (mode !== 'fixtures' &&
checks.check_docs_report),
publicationAllowed: mode === 'full',
checkRenderedLinks: !(files.length && files.every(file =>
/\.(css|scss|sass)$/.test(file))),
};
@@ -164,6 +174,8 @@ if (require.main === module) {
content_fixture: result.runContentFixture,
publish_fixture: result.runPublishFixture,
sampling_fixture: result.runSamplingFixture,
+ translation_tests: result.runTranslationTests,
+ docs_report: result.runDocsReport,
})
.map(([k, v]) => `${k}=${v}\n`)
.join(''),
diff --git a/scripts/i18n/verify-global-language.cjs
b/scripts/i18n/verify-global-language.cjs
index 761c0a06d871..986bce31ba98 100644
--- a/scripts/i18n/verify-global-language.cjs
+++ b/scripts/i18n/verify-global-language.cjs
@@ -48,7 +48,14 @@ function verifyContent(site, directory) {
}
return reports;
}
-async function verifyBrowser(site, {fixture = false, output =
'.i18n-build/browser-global-language.json'} = {}) {
+async function verifyBrowser(site, {fixture = false, manifests =
'.i18n-build/content-manifests',
+ ordinary = fixture ? 'sample-2/' : 'download/', output =
'.i18n-build/browser-global-language.json'} = {}) {
+ const manifest = JSON.parse(fs.readFileSync(path.join(manifests,
'ja.json')));
+ const ordinaryPage = manifest.pages.find(page => page.kind === 'page' &&
page.route === '/ja/' + ordinary);
+ assert(ordinaryPage, `Missing ordinary-page manifest entry:
/ja/${ordinary}`);
+ // The fixture deliberately exercises fallback. A real page may acquire a
+ // translation without changing its language-menu/navigation contract.
+ if (fixture && ordinary === 'sample-2/')
assert.equal(ordinaryPage.fallback, true);
const {chromium} = require('playwright');
const server = createServer(site);
await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));
@@ -72,10 +79,9 @@ async function verifyBrowser(site, {fixture = false, output
= '.i18n-build/brows
const page = await context.newPage();
page.on('pageerror', error => {errors.push(error.message);});
page.on('console', message => {if (/hydration|did not match|server
HTML/i.test(message.text())) errors.push(message.text());});
- const ordinary = fixture ? 'sample-2/' : 'download/';
await page.goto(origin + '/ja/' + ordinary + '?from=qa#usage');
await page.locator('.language-menu
a[href*="from=qa"]').first().waitFor({state: 'attached'});
- assert.equal(await
page.locator('[data-content-id]').first().getAttribute('lang'), 'en');
+ assert.equal(await
page.locator('[data-content-id]').first().getAttribute('lang'),
ordinaryPage.contentLocale);
const menu =
page.locator('.language-menu:not(.language-menu--mobile)');
await menu.locator('summary').click();
const english = menu.locator('a[lang="en-US"]');
@@ -151,8 +157,8 @@ async function verifyBrowser(site, {fixture = false, output
= '.i18n-build/brows
}
const noJs = await browser.newContext({javaScriptEnabled: false});
const noJsPage = await noJs.newPage(); await noJsPage.goto(origin +
'/ja/' + ordinary);
- assert.equal(await
noJsPage.locator('[data-content-id]').first().getAttribute('lang'), 'en');
- assert.equal(await noJsPage.locator('.page-fallback-notice').count(),
1);
+ assert.equal(await
noJsPage.locator('[data-content-id]').first().getAttribute('lang'),
ordinaryPage.contentLocale);
+ assert.equal(await noJsPage.locator('.page-fallback-notice').count(),
ordinaryPage.fallback ? 1 : 0);
assert.equal(await noJsPage.locator('.language-menu
a').first().getAttribute('href'), '/' + ordinary);
await noJs.close();
assert.deepEqual(errors, []);
@@ -165,8 +171,11 @@ async function main() {
const args = process.argv.slice(2); const value = (name, fallback) =>
args.includes(name) ? args[args.indexOf(name) + 1] : fallback;
const site = value('--site', 'build'), output = value('--output',
'.i18n-build/browser-global-language.json');
fs.mkdirSync(path.dirname(output), {recursive: true});
- const content = verifyContent(site, value('--manifests',
'.i18n-build/content-manifests'));
- const browser = await verifyBrowser(site, {fixture:
args.includes('--fixture'), output});
+ const manifests = value('--manifests', '.i18n-build/content-manifests');
+ const content = verifyContent(site, manifests);
+ const browser = await verifyBrowser(site, {fixture:
args.includes('--fixture'), manifests, output});
+ if (args.includes('--fixture'))
+ browser.translatedPage = await verifyBrowser(site, {fixture: true,
manifests, ordinary: 'sample-0/', output});
fs.writeFileSync(output, JSON.stringify({content, ...browser}, null, 2) +
'\n');
console.log(JSON.stringify({content, ...browser}));
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
