This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new 5bd5cbd4d3b [perf](website) Reduce multilingual publishing artifacts
(#4351)
5bd5cbd4d3b is described below
commit 5bd5cbd4d3ba4b96ac81eaa5a4a44b2057d1a1d2
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sun Oct 18 11:24:25 2026 +0100
[perf](website) Reduce multilingual publishing artifacts (#4351)
Complete multilingual publications duplicate images, fonts and PDFs
across locales and emit thousands of small legacy redirect pages. This
adds an optimized publishing command that shares resources and keeps the
existing document URLs and content.
Cron and manual deployment now use `yarn build:publish`. The bundler
generates root asset URLs and content-addressed binary filenames before
hashing, while search workers retain same-origin loading and
language-specific indexes. Assembly rejects conflicting asset filenames
and verifies resources before replacing the previous output.
Static binary aliases use non-cacheable HTTP 302 redirects, so
subsequent image updates can change their content hash. Exact Docusaurus
redirect templates become HTTP 301 rules with query and fragment
forwarding. Large alias families use guarded prefix rules; binary alias
tables live in their resource directories. Shared SVG logo artwork and
content-addressed blocking head scripts reduce repeated HTML while
retaining logo geometry and before-paint theme initialization. Normal
development builds keep their existing layout. Historical OSS objects
remain available for cached clients.
## Validation
Measured local **Dev-only, English/Chinese/Japanese** artifact:
| Metric | Before optimization | After optimization |
| --- | ---: | ---: |
| Raw artifact bytes | 3,459,794,290 | 1,494,368,877 |
| Files | 37,512 | 15,150 |
This reduces raw bytes by **56.8%** and files by **59.6%** in that
scope. It preserves 20,316 compatibility aliases and validates 47,477
compiled resource references. The Dev artifact has 34 root rewrite
rules; binary rules are scoped to static directories.
- 33 publication/build/configuration regression tests passed, including
real Apache coverage for Unicode, query/fragment handling, extension
aliases, real routes with HTML suffixes, asset updates, and atomic
failure behavior.
- 129 i18n regression tests passed.
- Real three-language Rspack and Webpack fixtures passed HTTP, Apache,
lazy-chunk, resource and search-worker browser checks.
- Three-language Dev build passed with separate sequential processes, a
4 GiB JS heap and SSR concurrency 2. Existing localized static
verification passed for 1,810 pages.
- Actual Dev browser verification passed 20 theme/viewport cases, three
localized documents, Japanese search, and 41 sampled compatibility
redirects, with no page or local-resource errors. As in the existing Dev
PoC, preview-only aliases serve each locale's own root Dev search
indexes for `/docs/dev/search-index*.json` requests.
- TypeScript reports the same 177 existing diagnostics as the clean
base, with no added diagnostics.
Publication infrastructure changes now run the complete optimized
artifact across all active versions in Build Check on CI and upload
measurement reports. No deployment is performed by this PR. Hosting
requirements and bounded local validation commands are documented in
`scripts/publish/README.md`.
## Versions
- [x] dev
- [x] 4.x
- [x] 3.x
- [x] 2.1 or older (not covered by version/language sync gate)
Applies to publication infrastructure for all active versions.
Documentation content, slugs, headings and sidebars are unchanged.
## Languages
- [x] Chinese
- [x] English
- [x] Japanese
## Docs Checklist
- [x] Checked by AI
- [x] Test Cases Built
- [x] Updated required version and language counterparts, or explained
why not
- [x] If only one language changed, confirmed whether source/translation
counterparts need sync
Shared infrastructure applies to every enabled locale, so no translated
documentation counterparts require edits.
Co-authored-by: morningman <[email protected]>
---
.github/workflows/build-check.yml | 22 ++
.github/workflows/cron-deploy-website.yml | 2 +-
.github/workflows/manual-deploy-website.yml | 2 +-
config/custom-docusaurus-plugin.js | 7 +-
config/ssrTemplate.js | 8 +-
docusaurus.config.js | 55 +----
package.json | 3 +
scripts/__tests__/build-locales.test.cjs | 18 ++
scripts/build-locales.cjs | 5 +
scripts/build-locales.md | 5 +
scripts/i18n/__tests__/t04.test.cjs | 12 +
scripts/i18n/build-scope.cjs | 8 +-
scripts/i18n/poc/config.test.cjs | 29 +++
scripts/publish/README.md | 89 +++++++
scripts/publish/__tests__/optimize.test.cjs | 158 +++++++++++++
scripts/publish/apache.cjs | 44 ++++
scripts/publish/bootstrap.cjs | 26 ++
scripts/publish/optimize.cjs | 261 +++++++++++++++++++++
scripts/publish/serve.cjs | 49 ++++
scripts/publish/shared-assets.cjs | 29 +++
scripts/publish/verify-fixture.cjs | 138 +++++++++++
scripts/publish/verify.cjs | 54 +++++
src/components/Icons/doris-logo-horizontal.tsx | 33 +--
.../brand-assets/doris-logo-horizontal-mark.svg | 1 +
static/js/brand-theme-bootstrap.js | 40 ++++
static/js/course-rail-bootstrap.js | 7 +
26 files changed, 1023 insertions(+), 82 deletions(-)
diff --git a/.github/workflows/build-check.yml
b/.github/workflows/build-check.yml
index d3275f70941..024f8ca37b2 100644
--- a/.github/workflows/build-check.yml
+++ b/.github/workflows/build-check.yml
@@ -114,8 +114,19 @@ jobs:
BASE_SHA=$(git merge-base origin/${{ github.base_ref }} HEAD)
node scripts/i18n/pr-preflight.cjs --base "$BASE_SHA"
+ - name: Publish artifact optimization tests
+ env:
+ NODE_OPTIONS: --max-old-space-size=2048
+ run: |
+ node --test scripts/publish/__tests__/*.test.cjs
+ BASE_SHA=$(git merge-base origin/${{ github.base_ref }} HEAD)
+ if git diff --name-only "$BASE_SHA" HEAD | grep -Eq
'^(scripts/publish/|config/(custom-docusaurus-plugin|ssrTemplate)\.js|scripts/build-locales\.cjs)';
then
+ node scripts/publish/verify-fixture.cjs
+ fi
+
- name: Build
env:
+ DORIS_OPTIMIZE_PUBLISH: ${{
steps.detect.outputs.optimize_publish }}
DOCS_VERSIONS: ${{ steps.detect.outputs.docs_versions }}
DORIS_ENABLE_LOCALES: ${{
steps.detect.outputs.enabled_locales }}
PWA_SERVICE_WORKER_URL: https://doris.apache.org/sw.js
@@ -147,6 +158,17 @@ jobs:
done
rm -rf build
+ - name: Upload publish artifact measurements
+ if: always() && steps.detect.outputs.optimize_publish == 'true'
+ uses: actions/upload-artifact@v4
+ with:
+ name: publish-artifact-measurements
+ path: |
+ .i18n-build/publish-result.json
+ .i18n-build/build-result.json
+ if-no-files-found: warn
+ retention-days: 14
+
- name: Docs changed link gate
continue-on-error: true
run: |
diff --git a/.github/workflows/cron-deploy-website.yml
b/.github/workflows/cron-deploy-website.yml
index b07af9f0552..b7b221b4f39 100644
--- a/.github/workflows/cron-deploy-website.yml
+++ b/.github/workflows/cron-deploy-website.yml
@@ -86,7 +86,7 @@ jobs:
yarn
node ./scripts/update_github_info.js
yarn key-features:generate
- PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js yarn
build:locales
+ PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js yarn
build:publish
touch build/.dummy
cp .asf-site.yaml ./build/.asf.yaml
cp versions.json ./build/
diff --git a/.github/workflows/manual-deploy-website.yml
b/.github/workflows/manual-deploy-website.yml
index beb28d85393..0a40bda4915 100644
--- a/.github/workflows/manual-deploy-website.yml
+++ b/.github/workflows/manual-deploy-website.yml
@@ -68,7 +68,7 @@ jobs:
yarn
node ./scripts/update_github_info.js
yarn key-features:generate
- PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js yarn
build:locales
+ PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js yarn
build:publish
touch build/.dummy
cp .asf-site.yaml ./build/.asf.yaml
cp versions.json ./build/
diff --git a/config/custom-docusaurus-plugin.js
b/config/custom-docusaurus-plugin.js
index 33a61c7997b..0d914b79461 100644
--- a/config/custom-docusaurus-plugin.js
+++ b/config/custom-docusaurus-plugin.js
@@ -1,4 +1,6 @@
const { getAssetOrigin } = require('./asset-origin');
+const {enabled, SharedAssetsPlugin} =
require('../scripts/publish/shared-assets.cjs');
+const {BootstrapAssetsPlugin} = require('../scripts/publish/bootstrap.cjs');
module.exports = function (context, options) {
return {
@@ -6,12 +8,15 @@ module.exports = function (context, options) {
configureWebpack(config, isServer, utils) {
// The resolved base URL also handles single-locale previews, where
// Docusaurus deliberately omits the locale prefix.
- const localePublicPath = context.siteConfig.baseUrl;
+ const localePublicPath = enabled() ? '/' :
context.siteConfig.baseUrl;
// An empty origin serves this build's assets locally. Unset keeps
// the production CDN. Worker bootstrap always stays same-origin.
const assetOrigin = getAssetOrigin();
return {
+ plugins: enabled() ? [new SharedAssetsPlugin(),
+ ...(!isServer ? [new
BootstrapAssetsPlugin(context.siteDir)] : [])] : [],
+ ...(enabled() && config.cache ?
{cache:{...config.cache,name:`${config.cache.name}-publish-v1`}} : {}),
output: {
...config.output,
publicPath: `${assetOrigin}${localePublicPath}`,
diff --git a/config/ssrTemplate.js b/config/ssrTemplate.js
index 7900744de83..be2b6f4b892 100644
--- a/config/ssrTemplate.js
+++ b/config/ssrTemplate.js
@@ -1,5 +1,7 @@
const { getAssetOrigin } = require('./asset-origin');
const assetOrigin = JSON.stringify(getAssetOrigin());
+const assetBase = require('../scripts/publish/shared-assets.cjs').enabled() ?
'/' : null;
+const assetBaseExpression = assetBase ? JSON.stringify(assetBase) :
'it.baseUrl';
module.exports = {
ssrTemplate: `<!DOCTYPE html>
@@ -16,10 +18,10 @@ module.exports = {
<%~ metaAttribute %>
<% }); %>
<% it.stylesheets.forEach((stylesheet) => { %>
- <link rel="stylesheet" href="<%= ${assetOrigin} %><%= it.baseUrl %><%=
stylesheet %>" />
+ <link rel="stylesheet" href="<%= ${assetOrigin} %><%=
${assetBaseExpression} %><%= stylesheet %>" />
<% }); %>
<% it.scripts.forEach((script) => { %>
- <link rel="preload" href="<%= ${assetOrigin} %><%= it.baseUrl %><%=
script %>" as="script">
+ <link rel="preload" href="<%= ${assetOrigin} %><%=
${assetBaseExpression} %><%= script %>" as="script">
<% }); %>
</head>
<body <%~ it.bodyAttributes %>>
@@ -28,7 +30,7 @@ module.exports = {
<%~ it.appHtml %>
</div>
<% it.scripts.forEach((script) => { %>
- <script src="<%= ${assetOrigin} %><%= it.baseUrl %><%= script
%>"></script>
+ <script src="<%= ${assetOrigin} %><%= ${assetBaseExpression} %><%=
script %>"></script>
<% }); %>
<%~ it.postBodyTags %>
</body>
diff --git a/docusaurus.config.js b/docusaurus.config.js
index 903490c7a09..ae7c2098759 100644
--- a/docusaurus.config.js
+++ b/docusaurus.config.js
@@ -34,57 +34,12 @@ const UTILITY_PATHS = new
Set(ENABLED_LOCALES.flatMap(locale => {
return [`${prefix}/search`, `${prefix}/slack`];
}));
-const BRAND_THEME_BOOTSTRAP = `(function () {
- var theme = 'doris';
- try {
- var storedTheme = localStorage.getItem('doris-brand-theme');
- var themes = ['doris', 'golden', 'blue', 'read', 'yellow-blue',
'purple', 'yellow-black', 'sky'];
- if (themes.indexOf(storedTheme) !== -1) theme = storedTheme;
- } catch (error) {}
- document.documentElement.setAttribute('data-brand-theme', theme);
-
- var kapaSelector =
'script[data-website-id="a5fb90df-217a-4097-95c0-80490220314b"]';
- var kapaColors = {
- doris: '#11A679',
- golden: '#7B2CBF',
- blue: '#2C2C34',
- read: '#121212',
- 'yellow-blue': '#0066FF',
- purple: '#7255A5',
- 'yellow-black': '#000000',
- sky: '#3778B0'
- };
- var kapaColor = kapaColors[theme];
- var observer;
- function syncKapaScript(node) {
- if (node.nodeType !== 1) return false;
- var script = node.matches && node.matches(kapaSelector)
- ? node
- : node.querySelector && node.querySelector(kapaSelector);
- if (!script) return false;
- script.setAttribute('data-project-color', kapaColor);
- if (observer) observer.disconnect();
- return true;
- }
- observer = new MutationObserver(function (mutations) {
- mutations.some(function (mutation) {
- return Array.prototype.some.call(mutation.addedNodes,
syncKapaScript);
- });
- });
- observer.observe(document.documentElement, { childList: true, subtree:
true });
- syncKapaScript(document.documentElement);
-}());`;
+const BRAND_THEME_BOOTSTRAP =
require('node:fs').readFileSync(require('node:path').join(__dirname,
'static/js/brand-theme-bootstrap.js'), 'utf8');
// Doris 101 remembers whether its module rail is collapsed. Applying the
stored
// choice before first paint keeps the rail from rendering expanded and then
// snapping shut once React mounts on every lesson navigation.
-const COURSE_RAIL_BOOTSTRAP = `(function () {
- try {
- if (localStorage.getItem('doris-101-rail-collapsed') === 'true') {
- document.documentElement.setAttribute('data-course-rail',
'collapsed');
- }
- } catch (error) {}
-}());`;
+const COURSE_RAIL_BOOTSTRAP =
require('node:fs').readFileSync(require('node:path').join(__dirname,
'static/js/course-rail-bootstrap.js'), 'utf8');
// Per-document last-update timestamps, generated from git history by
// scripts/last-update/generate.js. The 2-hourly Cron Deploy regenerates the
map
@@ -320,7 +275,11 @@ const config = {
localeConfigs: Object.fromEntries(enabledLocaleRecords()
.map(({code, label, htmlLang}) => [code, {label, htmlLang}])),
},
- headTags: [
+ headTags: require('./scripts/publish/shared-assets.cjs').enabled()
+ ?
require('./scripts/publish/bootstrap.cjs').scripts(__dirname).map(script => ({
+ // Blocking head scripts retain the before-paint theme/rail setup.
+ tagName:'script',
attributes:{src:require('./config/asset-origin').getAssetOrigin()+'/'+script.file},
+ })) : [
{
tagName: 'script',
attributes: {},
diff --git a/package.json b/package.json
index 4cbd0d79f91..7d6dc92a94e 100644
--- a/package.json
+++ b/package.json
@@ -10,6 +10,8 @@
"prestart": "yarn key-features:generate",
"build": "PWA_SERVICE_WORKER_URL=https://doris.apache.org/sw.js node
scripts/build-locales.cjs",
"build:locales": "node scripts/build-locales.cjs",
+ "build:publish": "DORIS_OPTIMIZE_PUBLISH=true node
scripts/build-locales.cjs",
+ "prebuild:publish": "yarn key-features:generate",
"prebuild": "yarn key-features:generate",
"build:version": "docusaurus set-versions && node
scripts/build-locales.cjs",
"prebuild:version": "yarn key-features:generate",
@@ -18,6 +20,7 @@
"deploy": "docusaurus deploy",
"clear": "docusaurus clear",
"serve": "docusaurus serve",
+ "serve:publish": "node scripts/publish/serve.cjs",
"build:zip": "rm -rf portal.zip && zip -r portal.zip build/**",
"generatePDF": "npx vitpress-generate-pdf -buildDirPath='./build/'
--firstDocPath='/docs/summary/basic-summary/'
--paginationSelector='.pagination-nav__link--next' --contentSelector='article'
--coverImage='https://cdn.selectdb.com/images/doris-logo-512.png'
--coverTitle='Apache Doris 1.1 (English)' --outputPDFFilename='Apache Doris 1.1
(English).pdf'",
"write-translations": "docusaurus write-translations --locale zh-CN",
diff --git a/scripts/__tests__/build-locales.test.cjs
b/scripts/__tests__/build-locales.test.cjs
index 0cd5657852d..0643450baec 100644
--- a/scripts/__tests__/build-locales.test.cjs
+++ b/scripts/__tests__/build-locales.test.cjs
@@ -121,3 +121,21 @@ test('an unsafe locale path is rejected before building
it', async t => {
await assert.rejects(buildLocales(data),/Unsafe locale/);
assert.equal(fs.readFileSync(path.join(data.outDir,'previous.txt'),'utf8'),'previous
complete site');
});
+
+test('failed optimized resource verification preserves the previous complete
artifact', async t => {
+ const previous = process.env.DORIS_OPTIMIZE_PUBLISH;
+ process.env.DORIS_OPTIMIZE_PUBLISH = 'true';
+ t.after(() => {
+ if (previous === undefined) delete process.env.DORIS_OPTIMIZE_PUBLISH;
+ else process.env.DORIS_OPTIMIZE_PUBLISH = previous;
+ });
+ const data = fixture(t,['en']);
+ const run = async args => {
+ const result = await data.run(args);
+ if (args[0] === 'build')
fs.writeFileSync(path.join(args[2],'index.html'),
+ '<script src=/assets/js/missing.js></script>');
+ return result;
+ };
+ await assert.rejects(buildLocales({...data,run}),/Missing compiled asset/);
+
assert.equal(fs.readFileSync(path.join(data.outDir,'previous.txt'),'utf8'),'previous
complete site');
+});
diff --git a/scripts/build-locales.cjs b/scripts/build-locales.cjs
index fbe72dcd487..34f2475747f 100644
--- a/scripts/build-locales.cjs
+++ b/scripts/build-locales.cjs
@@ -87,6 +87,11 @@ async function buildLocales({siteDir = process.cwd(), outDir
= 'build', locales:
fs.rmSync(destination,{recursive:true,force:true});
fs.renameSync(item.resultDir,destination);
}
+ if (require('./publish/shared-assets.cjs').enabled()) {
+ report.optimization =
require('./publish/optimize.cjs').optimize({site:assembled,
locales:report.locales});
+ report.optimization.verification =
require('./publish/verify.cjs').verify(assembled);
+
fs.writeFileSync(path.join(reportDir,'publish-result.json'),JSON.stringify(report.optimization,null,2)+'\n');
+ }
verifyStamp();
// Prepare every locale before touching the old complete artifact. Root
// controls come from the default locale; localized files stay beneath
diff --git a/scripts/build-locales.md b/scripts/build-locales.md
index b7adf105ec7..fc27191b02a 100644
--- a/scripts/build-locales.md
+++ b/scripts/build-locales.md
@@ -8,6 +8,11 @@ disabled unless `DORIS_ENABLE_LOCALES` lists them (CI) or
their own
no translation or deployment service is called. Both the locale list and each
locale's label and `htmlLang` come from `config/i18n.js`.
+Publishing workflows use `yarn build:publish`, which shares binary resources
+and webpack assets and compacts legacy redirects after assembly. See
+[publication artifacts](publish/README.md) for hosting requirements, previews,
+bounded local checks and size reports. Ordinary builds retain their layout.
+
```bash
# Local development: only the actual Dev docs version.
DOCS_VERSIONS=current DORIS_ASSET_ORIGIN='' yarn build:locales
diff --git a/scripts/i18n/__tests__/t04.test.cjs
b/scripts/i18n/__tests__/t04.test.cjs
index fc31a2a7f26..b2eb9ef6eae 100644
--- a/scripts/i18n/__tests__/t04.test.cjs
+++ b/scripts/i18n/__tests__/t04.test.cjs
@@ -43,6 +43,18 @@ test('PR build scope separates content locales, shared
inputs, and retired versi
assert.throws(() =>
buildScope(['translations/state/fr/docs.json'],versions,managed),/Unconfigured/);
});
+test('publication changes exercise the complete optimized artifact on CI', ()
=> {
+ for (const file of
['scripts/publish/optimize.cjs','scripts/build-locales.cjs',
+ 'config/ssrTemplate.js','static/js/brand-theme-bootstrap.js']) {
+ const scope = buildScope([file],versions,['ja']);
+ assert.equal(scope.optimize_publish,true,file);
+ assert.equal(scope.docs_versions,'',file);
+ assert.equal(scope.need_full_build,true,file);
+ assert.deepEqual(scope.locales,['en','zh-CN','ja']);
+ }
+
assert.equal(buildScope(['docs/example.md'],versions).optimize_publish,false);
+});
+
test('A localized index admits only current, reviewed and in-scope native
targets', () => {
const page =
(pluginId,version,status='current',indexable=true,locale='ja') =>
({pluginId,version,status,indexable,useTarget:true,locale,
permalink:`/${locale}/${pluginId}/${version}/${status}/${indexable}/`});
diff --git a/scripts/i18n/build-scope.cjs b/scripts/i18n/build-scope.cjs
index 00d401a08b5..945708cfdbd 100644
--- a/scripts/i18n/build-scope.cjs
+++ b/scripts/i18n/build-scope.cjs
@@ -39,7 +39,7 @@ function buildScope(files, versions, managed = []) {
else if (/^(blog|community|releasenotes|course)\//.test(file))
{languages.add('en'); build();}
else if (/^sidebars(Community|Releases|Course)\./.test(file)) shared();
else if (/^(src|config)\//.test(file) ||
['docusaurus.config.js','package.json','yarn.lock','tailwind.config.js','tsconfig.json'].includes(file)
- || /^scripts\/(key-features|last-update)\//.test(file) ||
/^scripts\/(patch-search-tokenize.js|build-locales.cjs|build-locale.cjs)$/.test(file)
+ || /^scripts\/(key-features|last-update|publish)\//.test(file) ||
/^scripts\/(patch-search-tokenize.js|build-locales.cjs|build-locale.cjs)$/.test(file)
||
/^scripts\/i18n\/(docs-plugin|status-plugin|state|structure)\.cjs$/.test(file))
shared();
// Translation configuration applies to every managed locale at once.
else if (file === 'translations/config.json') {managed.forEach(locale
=> languages.add(locale)); build();}
@@ -52,14 +52,20 @@ function buildScope(files, versions, managed = []) {
languages.add(match[1]); wanted.add(match[2]); json.add(file);
build();
}
else if (file === 'static/.htaccess') shared();
+ else if
(/^static\/js\/(brand-theme|course-rail)-bootstrap\.js$/.test(file)) shared();
else if
(/^(static|\.github|\.claude|\.agents|\.vscode|\.idea|scripts|website-quality-governance|translations|plan-doc)\//.test(file)
|| /\.(py|sh)$/.test(file)
||
/^(README.md|AGENTS.md|TOC.md|LICENSE|NOTICE|store_format_v3.md|cspell.json|author.yml|buildVersions.json|\.gitignore|\.prettier.*|\.eslintrc.*)$/.test(file))
{}
else {languages.add('en'); build();}
}
for (const version of wanted) if (!active.includes(version))
{wanted.delete(version); retired.add(version);}
+ const optimizePublish = files.some(file =>
/^(scripts\/publish\/|config\/(custom-docusaurus-plugin|ssrTemplate)\.js$|scripts\/build-locales\.cjs$|static\/js\/(brand-theme|course-rail)-bootstrap\.js$|\.github\/workflows\/(cron|manual)-deploy-website\.yml$)/.test(file));
+ // Publication changes need the same complete artifact as deployment, on
+ // the CI runner. Keep local checks scoped to fixtures and Dev versions.
+ if (optimizePublish) {tier = 'full'; shared();}
const selected = locales.filter(locale => languages.has(locale));
return {tier, skip_build: tier === 'skip', need_full_build: tier ===
'full',
+ optimize_publish: optimizePublish,
docs_versions: tier === 'full' ? '' : active.filter(version =>
wanted.has(version)).join(',') || versions[0] || 'current',
locales: selected,
// Only the opt-in locales have to be switched on for the build.
diff --git a/scripts/i18n/poc/config.test.cjs b/scripts/i18n/poc/config.test.cjs
index 272589bbe7b..63711e2997a 100644
--- a/scripts/i18n/poc/config.test.cjs
+++ b/scripts/i18n/poc/config.test.cjs
@@ -64,3 +64,32 @@ test('SSR bootstrap, preload and stylesheet URLs use the
same origin as webpack'
else process.env.DORIS_ASSET_ORIGIN = original;
}
});
+
+test('optimized publication shares CDN assets while workers remain on the page
origin', () => {
+ const previous = process.env.DORIS_OPTIMIZE_PUBLISH;
+ const origin = process.env.DORIS_ASSET_ORIGIN;
+ const templatePath = require.resolve('../../../config/ssrTemplate');
+ try {
+ process.env.DORIS_OPTIMIZE_PUBLISH = 'true';
+ process.env.DORIS_ASSET_ORIGIN = 'https://cdnd.selectdb.com';
+ for (const locale of ['/', '/zh-CN/', '/ja/']) {
+
assert.equal(paths(locale).publicPath,'https://cdnd.selectdb.com/');
+ assert.equal(paths(locale).workerPublicPath,'/');
+ }
+ delete require.cache[templatePath];
+ const html = require('eta').render(require(templatePath).ssrTemplate, {
+
htmlAttributes:'lang="ja"',bodyAttributes:'',version:'3.6.3',baseUrl:'/ja/',
+
headTags:'',metaAttributes:[],preBodyTags:'',postBodyTags:'',appHtml:'',noIndex:false,
+ stylesheets:['assets/css/main.css'],scripts:['assets/js/main.js'],
+ });
+
assert(html.includes('src="https://cdnd.selectdb.com/assets/js/main.js";'));
+
assert(html.includes('href="https://cdnd.selectdb.com/assets/css/main.css";'));
+ assert(!html.includes('/ja/assets/'));
+ } finally {
+ delete require.cache[templatePath];
+ if (previous === undefined) delete process.env.DORIS_OPTIMIZE_PUBLISH;
+ else process.env.DORIS_OPTIMIZE_PUBLISH = previous;
+ if (origin === undefined) delete process.env.DORIS_ASSET_ORIGIN;
+ else process.env.DORIS_ASSET_ORIGIN = origin;
+ }
+});
diff --git a/scripts/publish/README.md b/scripts/publish/README.md
new file mode 100644
index 00000000000..024a7548afa
--- /dev/null
+++ b/scripts/publish/README.md
@@ -0,0 +1,89 @@
+# Publication artifacts
+
+`yarn build:publish` builds the enabled locales in separate, sequential Node
+processes, then optimizes and verifies their assembled output before replacing
+the previous artifact. Cron and manual deployment use this entry point. Normal
+`yarn build`, `yarn build:locales`, development servers and locale-only checks
+retain their existing asset layout.
+
+## What changes
+
+- The bundler generates shared root asset URLs before calculating bundle
hashes.
+ Image, font and download loaders use content hashes instead of source names.
+ The assembler merges the locale asset directories, rejecting any filename
+ collision with different contents. JavaScript and CSS are never rewritten
+ after hashing. Search workers load from the same-origin root; their indexes
+ remain language-specific.
+- Identical static images, fonts and downloads share one file. Old public URLs
+ use non-cacheable temporary redirects to their canonical resource, so an
+ updated image cannot be pinned to an older content hash. Distinct compiled
asset filenames stay
+ present because the asset CDN does not interpret Apache rules.
+- Exact Docusaurus client-redirect templates become HTTP 301 rules. Large
+ `.html`/`.htm`, 3.x and release-note alias families use guarded prefix rules;
+ other aliases are grouped by destination. Real files retain priority, missing
+ destinations do not redirect through a generic rule, and query/fragment
+ forwarding follows the original template. Custom HTML such as the tracked
+ Slack redirect remains a page.
+ Binary compatibility rules live in their static-resource directories, so
+ normal pages and compiled asset requests do not parse the image alias tables.
+- Brand-theme and course-rail initialization use content-addressed blocking
+ head scripts during publication, preserving their before-paint behavior.
+ The cropped navbar logo is a shared SVG image with the same artwork and size.
+
+Document versions, translations, canonical URLs, heading IDs, navigation,
+search contents and search ranking do not change. The existing gzip delivery
+is retained. This does not delete historical objects from the OSS bucket;
+the existing incremental upload policy continues to preserve old cached assets.
+
+The optimized HTML site requires Apache with `mod_rewrite` and the generated
+root `.htaccess`, as used by the ASF website. The CDN serves actual compiled
+files at their new root URLs; it does not need to implement the compatibility
+redirects. An OSS object endpoint alone is not a replacement for the HTML host.
+Use `yarn serve:publish --site build --port 3000` for a local preview that
+understands the redirect manifest. A generic file server cannot reproduce
+removed legacy paths. `.publish/redirects.json` describes those redirects for
+preview and validation; it contains public URLs only.
+
+## Validation and memory
+
+Run the small real three-language fixture first. It exercises CSS/font/image
+loaders, language-specific bundles, lazy chunks, search workers, resources,
+redirect destinations and the default/subset assembly behavior:
+
+```bash
+node --test scripts/publish/__tests__/*.test.cjs
+NODE_OPTIONS=--max-old-space-size=2048 node scripts/publish/verify-fixture.cjs
+```
+
+With Playwright available separately through `NODE_PATH`, add `--browser`.
+`--apache` also checks the generated rules against a temporary loopback-only
+Apache instance. `--compare` builds an unoptimized fixture for size comparison;
+`--verify-existing` repeats HTTP/browser checks without rebuilding it.
+The fixture defaults to the site's Rspack mode; `--webpack` also verifies the
+non-faster Docusaurus build path.
+`DORIS_TEST_APACHE=true` enables the Apache regression test, including Unicode
+paths, query/fragment behavior, extension aliases and legacy version prefixes.
+
+Local website validation should stay within Dev, with bounded heap and SSR
+concurrency. Do not run concurrent builds in the same checkout:
+
+```bash
+yarn key-features:generate
+DOCS_VERSIONS=current DORIS_ENABLE_JA=true DORIS_ASSET_ORIGIN='' \
+ NODE_OPTIONS=--max-old-space-size=4096 DOCUSAURUS_SSR_CONCURRENCY=2 \
+ yarn build:publish --out-dir .i18n-build/dev-publish
+node scripts/publish/verify.cjs .i18n-build/dev-publish
+yarn serve:publish --site .i18n-build/dev-publish
+```
+
+`.i18n-build/publish-result.json` records before/after bytes, file counts,
+redirect pages, rule counts and verification. `.i18n-build/build-result.json`
+also records per-locale timing and build success. Counts describe raw files,
+not compressed network transfer or Git pack size. Source changes, invalid
+redirects, resource verification failures and asset collisions preserve the
+previous complete output.
+
+Changes to publication infrastructure make Build Check build every active
+version and enabled locale using the optimized mode on CI, with the existing
+8 GiB heap/swap configuration. It verifies localized metadata and uploads
+the measurement reports, without deploying the PR.
diff --git a/scripts/publish/__tests__/optimize.test.cjs
b/scripts/publish/__tests__/optimize.test.cjs
new file mode 100644
index 00000000000..3f3b65c664d
--- /dev/null
+++ b/scripts/publish/__tests__/optimize.test.cjs
@@ -0,0 +1,158 @@
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const os = require('node:os');
+const path = require('node:path');
+const {optimize} = require('../optimize.cjs');
+const redirect =
require('@docusaurus/plugin-client-redirects/lib/createRedirectPageContent').default;
+
+const locales = ['en','zh-CN','ja'].map(locale => ({locale,directory:locale
=== 'en' ? '.' : locale}));
+function fixture(t) {
+ const site = fs.mkdtempSync(path.join(os.tmpdir(),'doris-publish-test-'));
+ t.after(() => fs.rmSync(site,{recursive:true,force:true}));
+ const write = (name,content) => {
+ const file = path.join(site,name);
+ fs.mkdirSync(path.dirname(file),{recursive:true});
+ fs.writeFileSync(file,content);
+ };
+ write('.htaccess','# root controls\n');
+ for (const locale of locales) {
+ const prefix = locale.directory === '.' ? '' : locale.directory+'/';
+ write(prefix+'index.html','<main>Real homepage</main>');
+ write(prefix+'assets/shared/same.png',Buffer.alloc(65536,42));
+ write(prefix+'images/old.png',Buffer.alloc(65536,42));
+ write(prefix+'docs/dev/current/index.html','<main>Current
document</main>');
+
write(prefix+'docs/dev/current.html/index.html',redirect({toUrl:`/${prefix}docs/dev/current/`}));
+
write(prefix+'docs/dev/current.htm/index.html',redirect({toUrl:`/${prefix}docs/dev/current/`}));
+
write(prefix+'docs/dev/old/index.html',redirect({toUrl:`/${prefix}docs/dev/current/`}));
+ }
+ return {site,write};
+}
+
+test('shares binary assets without changing bundles, and retains every legacy
redirect target', t => {
+ const {site,write} = fixture(t);
+ write('ja/assets/js/ja.hash.js','const url="/assets/shared/same.png";');
+ write('assets/js/en.hash.js','const locale="en";');
+ write('ja/.htaccess','# root controls\n');
+ const result = optimize({site,locales});
+ assert(result.after.bytes < result.before.bytes / 3);
+ assert.equal(result.redirectPages,9);
+ assert(result.apacheRootRules < result.redirectPages);
+
assert.equal(fs.readFileSync(path.join(site,'assets/js/ja.hash.js'),'utf8'),'const
url="/assets/shared/same.png";');
+ assert(!fs.existsSync(path.join(site,'ja/assets')));
+ assert(!fs.existsSync(path.join(site,'ja/.htaccess')));
+ assert(!fs.existsSync(path.join(site,'images/old.png')));
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ assert(manifest.redirects.some(([from,to]) => from ===
'/ja/images/old.png' && to === '/assets/shared/same.png'));
+ assert(manifest.redirects.some(([from,to]) => from ===
'/ja/docs/dev/current.html/' && to === '/ja/docs/dev/current/'));
+ assert(fs.readFileSync(path.join(site,'.htaccess'),'utf8').endsWith('#
root controls\n'));
+});
+
+test('a locale-only artifact promotes its assets while preserving
language-specific differences', t => {
+ const {site,write} = fixture(t);
+ fs.rmSync(site,{recursive:true});
+ write('ja/index.html','<main>Japanese</main>');
+ write('ja/images/example.png','Japanese image');
+ write('zh-CN/images/example.png','Chinese image');
+ optimize({site,locales:locales.slice(1)});
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ for (const locale of ['ja','zh-CN']) {
+ const target = manifest.redirects.find(([from]) => from ===
`/${locale}/images/example.png`)[1];
+ assert.equal(fs.readFileSync(path.join(site,target),'utf8'),locale ===
'ja' ? 'Japanese image' : 'Chinese image');
+ }
+ assert(!fs.existsSync(path.join(site,'index.html')));
+});
+
+test('hash filename collisions fail instead of silently mixing languages', t
=> {
+ const {site,write} = fixture(t);
+ write('assets/js/same.hash.js','English');
+ write('ja/assets/js/same.hash.js','Japanese');
+ assert.throws(() => optimize({site,locales}),/filename collision/);
+});
+
+test('retains distinct compiled URLs even when their binary contents match', t
=> {
+ const {site,write} = fixture(t);
+ write('assets/shared/other-name.PNG',Buffer.alloc(65536,42));
+ optimize({site,locales});
+ assert(fs.existsSync(path.join(site,'assets/shared/same.png')));
+ assert(fs.existsSync(path.join(site,'assets/shared/other-name.PNG')));
+});
+
+test('verifies homepage redirects and unquoted resource attributes from the
faster minifier', t => {
+ const {site,write} = fixture(t);
+ write('vendors/index.html',redirect({toUrl:'/'}));
+ write('assets/js/entry.hash.js','console.log("entry");');
+ write('index.html','<script src=/assets/js/entry.hash.js></script>');
+ optimize({site,locales});
+ const result = require('../verify.cjs').verify(site);
+ assert.equal(result.compiledAssetReferences,1);
+});
+
+test('SVGs with relative references retain their original base URLs', t => {
+ const {site,write} = fixture(t);
+ for (const directory of ['one','two']) {
+ write(`images/${directory}/image.svg`,'<svg
xmlns="http://www.w3.org/2000/svg";><image href="nested.png"/></svg>');
+ write(`images/${directory}/nested.png`,directory);
+ }
+ optimize({site,locales});
+ for (const directory of ['one','two'])
assert(fs.existsSync(path.join(site,`images/${directory}/image.svg`)));
+});
+
+test('keeps custom HTML, preserves query/fragment semantics, and flattens
redirect chains', t => {
+ const {site,write} = fixture(t);
+
write('slack/index.html','<script>window.location.href="tracked"</script>');
+ write('first/index.html',redirect({toUrl:'/second/'}));
+
write('second/index.html',redirect({toUrl:'/docs/dev/current/?fixed=1#usage'}));
+ optimize({site,locales});
+ assert(fs.existsSync(path.join(site,'slack/index.html')));
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ assert.deepEqual(manifest.redirects.find(([from]) => from ===
'/first/'),['/first/','/docs/dev/current/?fixed=1#usage',false,301]);
+
assert(fs.readFileSync(path.join(site,'.htaccess'),'utf8').includes(',QSD]'));
+});
+
+test('rejects redirect cycles and artifact symlinks', t => {
+ const {site,write} = fixture(t);
+ write('one/index.html',redirect({toUrl:'/two/'}));
+ write('two/index.html',redirect({toUrl:'/one/'}));
+ assert.throws(() => optimize({site,locales}),/redirect cycle/);
+ fs.symlinkSync('/tmp',path.join(site,'outside'));
+ assert.throws(() => optimize({site,locales}),/symlink/);
+});
+
+test('Apache serves compacted legacy prefixes, Unicode paths, assets and
query/fragment redirects',
+ {skip:process.env.DORIS_TEST_APACHE !== 'true'}, async t => {
+ const {site,write} = fixture(t);
+ write('docs/3.x/prefix/index.html','<main>Legacy destination</main>');
+ write('docs/3.0/prefix/index.html',redirect({toUrl:'/docs/3.x/prefix/'}));
+ write('docs/prefix/index.html',redirect({toUrl:'/docs/3.x/prefix/'}));
+ write('releases/v4.0/release-4.0.1/index.html','<main>Release</main>');
+
write('docs/4.x/releasenotes/v4.0/release-4.0.1/index.html',redirect({toUrl:'/releases/v4.0/release-4.0.1/'}));
+
write('legacy-4.0/index.html',redirect({toUrl:'/docs/dev/current/?fixed=1#usage'}));
+ write('한글 alias/index.html',redirect({toUrl:'/docs/dev/current/#한글'}));
+ write('docs/한글/index.html','<main>Unicode target</main>');
+ write('docs/한글.html/index.html',redirect({toUrl:'/docs/한글/'}));
+ write('docs/real/index.html','<main>Short route</main>');
+ write('docs/real.html/index.html','<main>Distinct real HTML-suffix
route</main>');
+ optimize({site,locales});
+ const server = await require('../apache.cjs').apache(site);
+ t.after(() => server.close());
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ for (const [from,to,preserveQuery,status] of manifest.redirects) {
+ for (const source of from.endsWith('/') ?
[from,from.slice(0,-1),from+'index.html'] : [from]) {
+ const response = await
fetch(server.origin+source+'?probe=1',{redirect:'manual'});
+ assert.equal(response.status,status,source);
+ if (status === 302)
assert.equal(response.headers.get('cache-control'),'no-store');
+ const expected = new URL(to,server.origin);
+ if (preserveQuery) expected.search = '?probe=1';
+ const normalize = value => value.replace(/%[\da-f]{2}/ig,part =>
part.toUpperCase());
+ assert.equal(normalize(new
URL(response.headers.get('location'),server.origin).href),normalize(expected.href),source);
+ }
+ }
+ assert.equal((await fetch(server.origin+'/docs/3.0/missing/')).status,404);
+ const actualPage = await
fetch(server.origin+'/docs/real.html/',{redirect:'manual'});
+ assert.equal(actualPage.status,200);
+ assert((await actualPage.text()).includes('Distinct real HTML-suffix
route'));
+ assert.equal((await
fetch(server.origin+'/ja/assets/shared/same.png')).status,200);
+});
+
+module.exports = {fixture,locales};
diff --git a/scripts/publish/apache.cjs b/scripts/publish/apache.cjs
new file mode 100644
index 00000000000..3d22354a7f7
--- /dev/null
+++ b/scripts/publish/apache.cjs
@@ -0,0 +1,44 @@
+// A real Apache probe for the generated .htaccess, isolated on a loopback
port.
+const fs = require('node:fs');
+const path = require('node:path');
+const os = require('node:os');
+const net = require('node:net');
+const {spawn} = require('node:child_process');
+
+async function apache(site) {
+ const directory =
fs.mkdtempSync(path.join(os.tmpdir(),'doris-publish-apache-'));
+ const socket = net.createServer();
+ await new Promise(resolve => socket.listen(0,'127.0.0.1',resolve));
+ const port = socket.address().port;
+ await new Promise(resolve => socket.close(resolve));
+ const moduleDir = process.platform === 'darwin' ? '/usr/libexec/apache2' :
'/usr/lib/apache2/modules';
+ const modules =
['mpm_event','unixd','authz_core','authz_host','mime','dir','rewrite','headers'];
+ const config = [
+ `ServerRoot "${directory}"`, `DefaultRuntimeDir "${directory}"`,
`PidFile "${directory}/httpd.pid"`,
+ `Listen 127.0.0.1:${port}`, 'ServerName 127.0.0.1',
+ ...modules.filter(name =>
fs.existsSync(path.join(moduleDir,`mod_${name}.so`)))
+ .map(name => `LoadModule ${name}_module
"${moduleDir}/mod_${name}.so"`),
+ `User ${os.userInfo().username}`, `Group #${process.getgid()}`,
`ErrorLog "${directory}/error.log"`,
+ 'LogLevel warn', 'StartServers 1','ServerLimit 1','ThreadsPerChild
8','MaxRequestWorkers 8',
+ 'TypesConfig /etc/apache2/mime.types', `DocumentRoot
"${path.resolve(site)}"`,
+ `<Directory "${path.resolve(site)}">`, 'AllowOverride All','Require
all granted','DirectoryIndex index.html','</Directory>',
+ ].join('\n');
+ const configFile = path.join(directory,'httpd.conf');
+ fs.writeFileSync(configFile,config);
+ const binary = process.platform === 'darwin' ? '/usr/sbin/httpd' :
'/usr/sbin/apache2';
+ const child =
spawn(binary,['-f',configFile,'-D','FOREGROUND'],{stdio:['ignore','ignore','pipe']});
+ let stderr = '';
+ child.stderr.on('data',data => {stderr += data;});
+ const origin = `http://127.0.0.1:${port}`;
+ for (let attempt = 0; attempt < 100; attempt++) {
+ if (child.exitCode !== null) throw new Error(`Apache failed:
${stderr}`);
+ try {await fetch(origin);break;} catch {}
+ await new Promise(resolve => setTimeout(resolve,50));
+ }
+ return {origin,directory,async close() {
+ child.kill('SIGTERM');
+ if (child.exitCode === null) await new Promise(resolve =>
child.once('exit',resolve));
+ fs.rmSync(directory,{recursive:true,force:true});
+ }};
+}
+module.exports = {apache};
diff --git a/scripts/publish/bootstrap.cjs b/scripts/publish/bootstrap.cjs
new file mode 100644
index 00000000000..59ccfe6d42f
--- /dev/null
+++ b/scripts/publish/bootstrap.cjs
@@ -0,0 +1,26 @@
+const fs = require('node:fs');
+const path = require('node:path');
+const {createHash} = require('node:crypto');
+
+function scripts(siteDir) {
+ return ['brand-theme-bootstrap','course-rail-bootstrap'].map(name => {
+ const content =
fs.readFileSync(path.join(siteDir,`static/js/${name}.js`),'utf8');
+ const hash =
createHash('sha256').update(content).digest('hex').slice(0,16);
+ return {file:`assets/js/${name}.${hash}.js`,content};
+ });
+}
+
+class BootstrapAssetsPlugin {
+ constructor(siteDir) {this.siteDir = siteDir;}
+ apply(compiler) {
+ compiler.hooks.thisCompilation.tap('DorisBootstrapAssets', compilation
=> {
+ compilation.hooks.processAssets.tap({name:'DorisBootstrapAssets',
+
stage:compiler.webpack.Compilation.PROCESS_ASSETS_STAGE_ADDITIONAL}, () => {
+ for (const script of scripts(this.siteDir))
compilation.emitAsset(script.file,
+ new compiler.webpack.sources.RawSource(script.content));
+ });
+ });
+ }
+}
+
+module.exports = {scripts,BootstrapAssetsPlugin};
diff --git a/scripts/publish/optimize.cjs b/scripts/publish/optimize.cjs
new file mode 100644
index 00000000000..29d96c66207
--- /dev/null
+++ b/scripts/publish/optimize.cjs
@@ -0,0 +1,261 @@
+const fs = require('node:fs');
+const path = require('node:path');
+const {createHash} = require('node:crypto');
+const assert = require('node:assert/strict');
+
+const binary =
/\.(?:png|jpe?g|gif|webp|avif|svg|ico|ttf|otf|woff2?|eot|pdf|zip|docx?|xlsx?|mp[34]|webm|wav|ogg)$/i;
+const begin = '# BEGIN DORIS PUBLISH OPTIMIZATION';
+const end = '# END DORIS PUBLISH OPTIMIZATION';
+const escapeRegex = value => value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+const digest = file =>
createHash('sha256').update(fs.readFileSync(file)).digest('hex');
+
+function inventory(root) {
+ const files = [];
+ function walk(directory) {
+ for (const entry of fs.readdirSync(directory,
{withFileTypes:true}).sort((a,b) => a.name.localeCompare(b.name))) {
+ const file = path.join(directory, entry.name);
+ assert(!entry.isSymbolicLink(), `Publish artifact contains a
symlink: ${file}`);
+ if (entry.isDirectory()) walk(file);
+ else if (entry.isFile()) files.push({file,
relative:path.relative(root,file).split(path.sep).join('/'),
size:fs.statSync(file).size});
+ }
+ }
+ walk(root);
+ return files;
+}
+
+function optimize({site, locales, reportFile}) {
+ site = path.resolve(site);
+ const before = inventory(site);
+ const prefixes = locales.filter(locale => locale.directory !==
'.').map(locale => locale.directory);
+ const stripLocale = relative => {
+ const prefix = prefixes.find(value =>
relative.startsWith(`${value}/`));
+ return prefix ? relative.slice(prefix.length + 1) : relative;
+ };
+ const redirects = new Map();
+ const controls = path.join(site,'.htaccess');
+ for (const prefix of prefixes) {
+ const copy = path.join(site,prefix,'.htaccess');
+ if (!fs.existsSync(copy)) continue;
+ if (!fs.existsSync(controls)) fs.copyFileSync(copy,controls);
+
assert.equal(fs.readFileSync(copy,'utf8'),fs.readFileSync(controls,'utf8'),
+ `Locale has different Apache controls: ${prefix}`);
+ fs.unlinkSync(copy);
+ }
+ const url = relative => '/' +
relative.split('/').map(encodeURIComponent).join('/');
+ const add = (from, to, preserveQuery = true, status = 301) => {
+ assert(from.startsWith('/') && !/[\r\n]/.test(from + to), 'Unsafe
publication redirect');
+ assert(!redirects.has(from) || redirects.get(from).to === to,
`Conflicting publication redirect: ${from}`);
+ redirects.set(from, {to, preserveQuery,status});
+ };
+ let sharedFiles = 0, redirectPages = 0;
+
+ // All webpack runtimes already reference the root pool, including lazy
+ // chunks and same-origin search workers. Never rewrite a hashed bundle.
+ for (const item of before) {
+ const relative = stripLocale(item.relative);
+ if (relative === item.relative || !relative.startsWith('assets/'))
continue;
+ const target = path.join(site, relative);
+ if (fs.existsSync(target)) {
+ assert.equal(digest(item.file), digest(target), `Shared asset
filename collision: ${relative}`);
+ fs.unlinkSync(item.file);
+ } else {
+ fs.mkdirSync(path.dirname(target), {recursive:true});
+ fs.renameSync(item.file,target);
+ }
+ sharedFiles++;
+ }
+
+ // Keep webpack's content-addressed URL when a static file also passed
+ // through a loader. Public legacy URLs remain HTTP redirects to that file.
+ const groups = new Map();
+ for (const item of inventory(site).filter(item =>
binary.test(item.relative))) {
+ // A standalone SVG may resolve other resources relative to its own
+ // location. Identical bytes do not imply interchangeable base URLs.
+ if (/\.svg$/i.test(item.relative) &&
/(?:href\s*=\s*["'](?!#|data:)|url\(\s*["']?(?!#|data:))/i
+ .test(fs.readFileSync(item.file,'utf8'))) continue;
+ const key =
`${digest(item.file)}:${path.extname(item.relative).toLowerCase()}`;
+ if (!groups.has(key)) groups.set(key, []);
+ groups.get(key).push(item);
+ }
+ for (const items of groups.values()) {
+ const rank = item => item.relative.startsWith('assets/') ? 0 :
stripLocale(item.relative) === item.relative ? 1 : 2;
+ items.sort((a,b) => rank(a)-rank(b) ||
a.relative.length-b.relative.length || a.relative.localeCompare(b.relative));
+ let canonical = items[0];
+ if (rank(canonical) === 2) {
+ let relative = stripLocale(canonical.relative);
+ if (fs.existsSync(path.join(site,relative))) relative =
`assets/shared/${digest(canonical.file)}${path.extname(relative)}`;
+ const file = path.join(site,relative);
+ fs.mkdirSync(path.dirname(file),{recursive:true});
+ fs.copyFileSync(canonical.file,file);
+ canonical = {relative,file};
+ }
+ for (const item of items) {
+ if (item.relative === canonical.relative) continue;
+ // Distinct compiled URLs may be used on the CDN, which does not
+ // interpret Apache aliases. Their content hashes are established
+ // by webpack; retain every distinct root asset filename.
+ if (item.relative.startsWith('assets/')) continue;
+ add(url(item.relative),url(canonical.relative),true,302);
+ fs.unlinkSync(item.file);
+ sharedFiles++;
+ }
+ }
+
+ const createRedirect =
require('@docusaurus/plugin-client-redirects/lib/createRedirectPageContent').default;
+ const cheerio = require('cheerio');
+ for (const item of inventory(site)) {
+ if (path.basename(item.file) === '.DS_Store') {
+ fs.unlinkSync(item.file);
+ continue;
+ }
+ if (!item.relative.endsWith('/index.html') || item.size > 8192)
continue;
+ const content = fs.readFileSync(item.file,'utf8');
+ if (!content.includes('window.location.href')) continue;
+ const canonical =
cheerio.load(content)('link[rel="canonical"]').attr('href');
+ if (!canonical) continue;
+ let target;
+ try { target = decodeURI(canonical); } catch { continue; }
+ // Only recognize the pinned plugin's exact template. Tracking pages,
+ // custom redirects and arbitrary small HTML pages stay untouched.
+ if (createRedirect({toUrl:target}) !== content) continue;
+ const from = url(item.relative.slice(0,-'index.html'.length));
+ const parsed = new URL(target,'https://doris.apache.org');
+ add(from,canonical,!parsed.search && !parsed.hash);
+ fs.unlinkSync(item.file);
+ redirectPages++;
+ }
+
+ // Resolve chains once, then reject cycles before committing the artifact.
+ for (const [from, item] of redirects) {
+ const seen = new Set([from]);
+ while (redirects.has(item.to)) {
+ assert(!seen.has(item.to), `Publication redirect cycle: ${from}`);
+ seen.add(item.to);
+ const next = redirects.get(item.to);
+ item.to = next.to;
+ item.preserveQuery = item.preserveQuery && next.preserveQuery;
+ if (next.status === 302) item.status = 302;
+ }
+ }
+
+ const apache = writeApache({site,locales,redirects});
+ fs.mkdirSync(path.join(site,'.publish'),{recursive:true});
+
fs.writeFileSync(path.join(site,'.publish/redirects.json'),JSON.stringify({schemaVersion:1,
+ assetPrefixes:prefixes.map(prefix => `/${prefix}/assets/`),
+ redirects:[...redirects].map(([from,item]) =>
[from,item.to,item.preserveQuery,item.status])}));
+ function clean(directory) {
+ for (const entry of fs.readdirSync(directory,{withFileTypes:true})) {
+ if (!entry.isDirectory()) continue;
+ const child = path.join(directory,entry.name);
+ clean(child);
+ if (!fs.readdirSync(child).length) fs.rmdirSync(child);
+ }
+ }
+ clean(site);
+ const after = inventory(site);
+ const result = {before:{files:before.length,bytes:before.reduce((n,item)
=> n+item.size,0)},
+ after:{files:after.length,bytes:after.reduce((n,item) =>
n+item.size,0)},sharedFiles,redirectPages,
+ apacheRules:apache.total,apacheRootRules:apache.root};
+ if (reportFile) {
+ fs.mkdirSync(path.dirname(reportFile),{recursive:true});
+ fs.writeFileSync(reportFile,JSON.stringify(result,null,2)+'\n');
+ }
+ return result;
+}
+
+module.exports = {optimize,inventory,writeApache};
+
+function writeApache({site,locales,redirects,replaceExisting = false}) {
+ const prefixes = locales.filter(locale => locale.directory !==
'.').map(locale => locale.directory);
+ const controls = path.join(site,'.htaccess');
+ const rules = [begin, '<IfModule mod_rewrite.c>', 'RewriteEngine On',
+ // Existing files must retain priority over every compatibility alias.
+ 'RewriteCond %{REQUEST_FILENAME} -f [OR]', 'RewriteCond
%{REQUEST_FILENAME}/index.html -f',
+ 'RewriteRule ^ - [END]'];
+ const generic = [];
+ const extensionAliases = [...redirects].filter(([from,item]) =>
from.replace(/\.html?\/$/,'/') === item.to);
+ if (extensionAliases.length) {
+ rules.push('RewriteCond %{DOCUMENT_ROOT}/$1/index.html -f',
+ 'RewriteRule "^(.+?)\\.html?(?:/index\\.html|/)?$" "/$1/"
[R=301,L]');
+ generic.push(([from,item]) => from.replace(/\.html?\/$/,'/') ===
item.to);
+ }
+ for (const prefix of prefixes) {
+ rules.push('RewriteCond %{DOCUMENT_ROOT}/$1 -f',
+ `RewriteRule "^${escapeRegex(prefix)}/(assets/.+)$" "/$1"
[R=302,L,E=DORIS_ASSET_REDIRECT:1]`);
+ }
+ // Compress the large legacy prefix families into guarded rules. Only a
+ // real built destination can redirect an otherwise missing source URL.
+ for (const locale of locales) {
+ const prefix = locale.directory === '.' ? '' : `${locale.directory}/`;
+ for (const [source,target] of
[[`${prefix}docs/3.0/`,`${prefix}docs/3.x/`],
+ [`${prefix}docs/`,`${prefix}docs/3.x/`]]) {
+ const matches = ([from,item]) => from.startsWith('/'+source)
+ && item.to === '/'+target+from.slice(source.length+1);
+ if ([...redirects].some(matches)) {
+ rules.push(`RewriteCond
%{DOCUMENT_ROOT}/${target}$1/index.html -f`,
+ `RewriteRule
"^${escapeRegex(source)}(.+?)(?:/index\\.html|/)?$" "/${target}$1/" [R=301,L]`);
+ generic.push(matches);
+ }
+ }
+ const release = ([from,item]) => {
+ const match = from.match(new
RegExp(`^/${escapeRegex(prefix)}docs/(?:4\\.x|3\\.x|2\\.1|2\\.0|1\\.2|4\\.0|3\\.1|3\\.0|dev)/releasenotes/(.+)$`));
+ return match && item.to === `/${prefix}releases/${match[1]}`;
+ };
+ if ([...redirects].some(release)) {
+ rules.push(`RewriteCond
%{DOCUMENT_ROOT}/${prefix}releases/$1/index.html -f`,
+ `RewriteRule
"^${escapeRegex(prefix)}docs/(?:4\\.x|3\\.x|2\\.1|2\\.0|1\\.2|4\\.0|3\\.1|3\\.0|dev)/releasenotes/(.+?)(?:/index\\.html|/)?$"
"/${prefix}releases/$1/" [R=301,L]`);
+ generic.push(release);
+ }
+ }
+ const exact = new Map();
+ for (const entry of redirects) {
+ if (generic.some(matches => matches(entry))) continue;
+ const [from,item] = entry;
+ const decoded = decodeURIComponent(from).replace(/^\//,'');
+ const localePrefix = prefixes.find(prefix =>
decoded.startsWith(prefix+'/'));
+ const relative = localePrefix ? decoded.slice(localePrefix.length+1) :
decoded;
+ const first = relative.split('/')[0];
+ // Keep large binary alias tables out of ordinary HTML/JS requests.
+ // These namespaces contain static resources, not content-plugin
routes.
+ const directory = item.status === 302 &&
['images','fonts','pdf','files'].includes(first)
+ ? (localePrefix ? localePrefix+'/' : '')+first : '';
+ const key = JSON.stringify({item,directory});
+ if (!exact.has(key)) exact.set(key, {item,directory,from:[]});
+ exact.get(key).from.push({path:(directory ?
decoded.slice(directory.length+1) : decoded).replace(/\/$/,''),
+ directory:from.endsWith('/')});
+ }
+ const assetRules = new Map();
+ for (const {item,from,directory} of exact.values()) {
+ // Quote and escape config syntax, while leaving the already encoded
+ // redirect target intact (NE avoids double encoding it).
+ const target =
item.to.replace(/\\/g,'\\\\').replace(/"/g,'\\"').replace(/\$/g,'\\$').replace(/%/g,'\\%');
+ const pattern = from.map(value => escapeRegex(value.path)
+ + (value.directory ? '(?:/index\\.html|/)?' :
'')).join('|').replace(/"/g,'\\"');
+ const rule = `RewriteRule "^(?:${pattern})$" "${target}"
[R=${item.status},L,NE${item.preserveQuery ? '' : ',QSD'}${item.status === 302
? ',E=DORIS_ASSET_REDIRECT:1' : ''}]`;
+ if (directory) {
+ if (!assetRules.has(directory)) assetRules.set(directory,[]);
+ assetRules.get(directory).push(rule);
+ } else rules.push(rule);
+ }
+ rules.push('</IfModule>','<IfModule mod_headers.c>',
+ 'Header always set Cache-Control "no-store"
env=DORIS_ASSET_REDIRECT','</IfModule>',end);
+ let original = fs.existsSync(controls) ? fs.readFileSync(controls,'utf8')
: '';
+ if (original.includes(begin)) {
+ assert(replaceExisting, 'Artifact was already optimized');
+ const finish = original.indexOf(end)+end.length;
+ assert(finish >= end.length, 'Incomplete publication controls');
+ original =
original.slice(0,original.indexOf(begin))+original.slice(finish).replace(/^\n+/,'');
+ }
+ fs.writeFileSync(controls,rules.join('\n')+'\n\n'+original);
+ for (const [directory,lines] of assetRules) {
+ const file = path.join(site,directory,'.htaccess');
+ assert(!fs.existsSync(file) || (replaceExisting &&
fs.readFileSync(file,'utf8').startsWith(begin)),
+ `Cannot replace existing asset controls: ${directory}`);
+ fs.mkdirSync(path.dirname(file),{recursive:true});
+ fs.writeFileSync(file,[begin,'<IfModule mod_rewrite.c>','RewriteEngine
On',
+ 'RewriteOptions
IgnoreInherit',...lines,'</IfModule>',end,''].join('\n'));
+ }
+ return {root:rules.filter(line => line.startsWith('RewriteRule')).length,
+ total:rules.filter(line => line.startsWith('RewriteRule')).length
+ + [...assetRules.values()].reduce((n,lines) => n+lines.length,0)};
+}
diff --git a/scripts/publish/serve.cjs b/scripts/publish/serve.cjs
new file mode 100644
index 00000000000..47a1b500679
--- /dev/null
+++ b/scripts/publish/serve.cjs
@@ -0,0 +1,49 @@
+const fs = require('node:fs');
+const path = require('node:path');
+const http = require('node:http');
+
+function createServer(site) {
+ site = path.resolve(site);
+ const file = path.join(site,'.publish/redirects.json');
+ const manifest = fs.existsSync(file) ?
JSON.parse(fs.readFileSync(file,'utf8')) : {redirects:[],assetPrefixes:[]};
+ const redirects = new
Map(manifest.redirects.map(([from,to,preserveQuery,status = 301]) =>
[decodeURIComponent(from),{to,preserveQuery,status}]));
+ return http.createServer((request,response) => {
+ try {
+ const url = new URL(request.url,'http://preview');
+ const lookup =
decodeURIComponent(url.pathname).replace(/\/index\.html$/,'/');
+ let redirect = redirects.get(lookup) ?? redirects.get(lookup+'/');
+ if (!redirect) {
+ const prefix = manifest.assetPrefixes.find(prefix =>
url.pathname.startsWith(prefix));
+ if (prefix) {
+ const target =
'/assets/'+url.pathname.slice(prefix.length);
+ if
(fs.existsSync(path.join(site,decodeURIComponent(target)))) redirect =
{to:target,preserveQuery:true,status:302};
+ }
+ }
+ if (redirect) {
+ const target = new URL(redirect.to,'http://preview');
+ if (redirect.preserveQuery) target.search = url.search;
+ const location = target.origin === 'http://preview' ?
target.pathname+target.search+target.hash : target.href;
+
response.writeHead(redirect.status,{Location:location,...(redirect.status ===
302 ? {'Cache-Control':'no-store'} : {})});
+ response.end(); return;
+ }
+ const requested =
path.resolve(site,'.'+decodeURIComponent(url.pathname));
+ if (!requested.startsWith(site+path.sep) && requested !== site)
throw new Error('Outside site');
+ const target = fs.existsSync(requested) &&
fs.statSync(requested).isDirectory() ? path.join(requested,'index.html') :
requested;
+ if (!fs.existsSync(target) || !fs.statSync(target).isFile())
{response.writeHead(404);response.end('Not found');return;}
+ const types = {'.html':'text/html;
charset=utf-8','.js':'application/javascript','.css':'text/css',
+
'.json':'application/json','.svg':'image/svg+xml','.png':'image/png','.jpg':'image/jpeg',
+
'.jpeg':'image/jpeg','.pdf':'application/pdf','.ttf':'font/ttf','.woff2':'font/woff2'};
+ response.writeHead(200,{'Content-Type':types[path.extname(target)]
?? 'application/octet-stream',
+ 'Content-Length':fs.statSync(target).size});
+ fs.createReadStream(target).pipe(response);
+ } catch {response.writeHead(400);response.end('Invalid request');}
+ });
+}
+
+if (require.main === module) {
+ const args = process.argv.slice(2);
+ const value = (name,fallback) => args.includes(name) ?
args[args.indexOf(name)+1] : fallback;
+ const server = createServer(value('--site','build'));
+ server.listen(Number(value('--port','3000')),'127.0.0.1',() =>
console.log(`Preview: http://127.0.0.1:${server.address().port}`));
+}
+module.exports = {createServer};
diff --git a/scripts/publish/shared-assets.cjs
b/scripts/publish/shared-assets.cjs
new file mode 100644
index 00000000000..251864ca766
--- /dev/null
+++ b/scripts/publish/shared-assets.cjs
@@ -0,0 +1,29 @@
+const {parseQuery} = require('loader-utils');
+
+const enabled = () => process.env.DORIS_OPTIMIZE_PUBLISH === 'true';
+
+// Configure loaders before code generation: changing URLs in already hashed
+// bundles would make an old cache entry valid for different file contents.
+class SharedAssetsPlugin {
+ apply(compiler) {
+ compiler.hooks.compilation.tap('DorisSharedAssets', compilation => {
+ const {NormalModule} = compiler.rspack ?? compiler.webpack;
+ // Rspack 1.x does not expose loaders through afterResolve. This
+ // hook runs before either bundler executes its JS loaders.
+
NormalModule.getCompilationHooks(compilation).loader.tap('DorisSharedAssets',
(context,module) => {
+ // Webpack initializes context.loaders after this hook, while
+ // Rspack has already constructed its loader-runner objects.
+ for (const item of context.loaders ?? module.loaders) {
+ if
(!/[\\/]node_modules[\\/](?:file-loader|url-loader)[\\/]/.test(item.path ??
item.loader)) continue;
+ const raw = item.options ?? item.query?.replace(/^\?/,'');
+ const options = typeof raw === 'string' ?
parseQuery('?'+raw) : raw ?? {};
+ item.options = {...options, name:
'assets/shared/[contenthash].[ext]'};
+ if ('query' in item) item.query =
'?'+JSON.stringify(item.options);
+ context.addBuildDependency(__filename);
+ }
+ });
+ });
+ }
+}
+
+module.exports = {enabled, SharedAssetsPlugin};
diff --git a/scripts/publish/verify-fixture.cjs
b/scripts/publish/verify-fixture.cjs
new file mode 100644
index 00000000000..2730675dbc3
--- /dev/null
+++ b/scripts/publish/verify-fixture.cjs
@@ -0,0 +1,138 @@
+const fs = require('node:fs');
+const path = require('node:path');
+const assert = require('node:assert/strict');
+const {createHash} = require('node:crypto');
+const {buildLocales} = require('../build-locales.cjs');
+const {createServer} = require('./serve.cjs');
+const {inventory} = require('./optimize.cjs');
+
+const root = path.resolve(__dirname,'../..');
+const fixture = path.join(root,'.i18n-build/publish-fixture');
+const locales = ['en','zh-CN','ja'];
+const hash = content => createHash('sha256').update(content).digest('hex');
+
+async function main() {
+ const verifyOnly = process.argv.includes('--verify-existing');
+ const image =
Buffer.concat([Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+jivsAAAAASUVORK5CYII=','base64'),Buffer.alloc(65536)]);
+ if (!verifyOnly) {
+ fs.rmSync(fixture,{recursive:true,force:true});
+ const write = (file,content) => {
+ fs.mkdirSync(path.dirname(path.join(fixture,file)),{recursive:true});
+ fs.writeFileSync(path.join(fixture,file),content);
+ };
+ write('package.json','{"name":"publish-fixture","private":true}');
+
fs.symlinkSync(path.join(root,'node_modules'),path.join(fixture,'node_modules'));
+ for (const name of ['brand-theme-bootstrap','course-rail-bootstrap'])
write(`static/js/${name}.js`,
+ fs.readFileSync(path.join(root,`static/js/${name}.js`)));
+
write('static/.htaccess',fs.readFileSync(path.join(root,'static/.htaccess')));
+ write('static/images/shared.png',image);
+ write('static/pdf/reference.pdf','%PDF-1.4\nFixture reference document\n');
+
write('static/fonts/inter.ttf',fs.readFileSync(path.join(root,'static/fonts/Inter-VariableFont_opsz,wght.ttf')));
+ write('static/fixture.css','@font-face
{font-family:Fixture;src:url("/fonts/inter.ttf")} main {font-family:Fixture}');
+ const document = '# Example\n\nFixtureSQL documents Apache
Doris.\n\n\n\n[Reference
PDF](/pdf/reference.pdf)\n';
+ write('docs/intro.md',document);
+ for (const locale of locales.slice(1))
write(`i18n/${locale}/docusaurus-plugin-content-docs/current/intro.md`,
+ document.replace('documents',`${locale} documents`));
+ write('src/pages/index.jsx',`import React from 'react';
+import useDocusaurusContext from '@docusaurus/useDocusaurusContext';
+import {searchByWorker} from
'@yang1666204/docusaurus-search-local/dist/client/client/theme/searchByWorker';
+export default function Home() {const [value,setValue]=React.useState('');
+const {siteConfig:{baseUrl},i18n:{currentLocale}}=useDocusaurusContext();
+return <main><button onClick={async()=>setValue((await
import('./_lazy')).default)}>Load module</button>
+<button onClick={async()=>{const results=await
searchByWorker(baseUrl,undefined,'FixtureSQL',10);
+setValue(results.length>0&&results.every(r=>r.document.u.startsWith(baseUrl+'docs/'))?'SEARCH_OK_'+currentLocale:'SEARCH_WRONG')}}>Search
module</button><p>{value}</p></main>}`);
+ write('src/pages/_lazy.js','export default "LAZY_OK";');
+ write('docusaurus.config.js',`const shared =
require(${JSON.stringify(path.join(root,'scripts/publish/shared-assets.cjs'))});
+const {scripts} =
require(${JSON.stringify(path.join(root,'scripts/publish/bootstrap.cjs'))});
+module.exports = {title:'Publish
fixture',url:'https://doris.apache.org',baseUrl:'/',trailingSlash:true,
+future:{experimental_faster:${!process.argv.includes('--webpack')}},
+i18n:{defaultLocale:'en',locales:${JSON.stringify(locales)}},
+...require(${JSON.stringify(path.join(root,'config/ssrTemplate.js'))}),
+headTags:shared.enabled()?scripts(__dirname).map(s=>({tagName:'script',attributes:{src:'/'+s.file}})):[],
+presets:[['classic',{docs:{path:'docs',sidebarPath:false},blog:false,theme:{customCss:'./static/fixture.css'}}]],
+plugins:[${JSON.stringify(path.join(root,'config/custom-docusaurus-plugin.js'))},
+['@docusaurus/plugin-client-redirects',{fromExtensions:['html','htm'],redirects:[{from:'/legacy',to:'/docs/intro/'}]}]],
+themes:[['@yang1666204/docusaurus-search-local',{hashed:true,language:['en'],docsRouteBasePath:'docs',indexBlog:false}]],
+};`);
+ }
+ let baseline;
+ process.env.DORIS_ASSET_ORIGIN = '';
+ if (!verifyOnly && process.argv.includes('--compare')) {
+ delete process.env.DORIS_OPTIMIZE_PUBLISH;
+ await buildLocales({siteDir:fixture,outDir:'baseline'});
+ const files = inventory(path.join(fixture,'baseline'));
+ baseline = {files:files.length,bytes:files.reduce((n,item) =>
n+item.size,0)};
+ }
+ process.env.DORIS_OPTIMIZE_PUBLISH = 'true';
+ const report = verifyOnly ?
JSON.parse(fs.readFileSync(path.join(fixture,'.i18n-build/build-result.json'),'utf8'))
+ : await buildLocales({siteDir:fixture,outDir:'build'});
+ if (verifyOnly && fs.existsSync(path.join(fixture,'baseline'))) {
+ const files = inventory(path.join(fixture,'baseline'));
+ baseline = {files:files.length,bytes:files.reduce((n,item) =>
n+item.size,0)};
+ }
+ const site = path.join(fixture,'build');
+ if (verifyOnly) report.optimization.verification =
require('./verify.cjs').verify(site);
+ const server = createServer(site);
+ await new Promise(resolve => server.listen(0,'127.0.0.1',resolve));
+ const previewOrigin = `http://127.0.0.1:${server.address().port}`;
+ let apache;
+ try {
+ if (process.argv.includes('--apache')) apache = await
require('./apache.cjs').apache(site);
+ for (const origin of [previewOrigin,...(apache ? [apache.origin] :
[])]) {
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ for (const [from,to,preserveQuery,status = 301] of
manifest.redirects) {
+ const response = await
fetch(origin+from+'?probe=1',{redirect:'manual'});
+ assert.equal(response.status,status,`${origin}${from}`);
+ const location = new
URL(response.headers.get('location'),origin);
+ const expected = new URL(to,origin);
+ if (preserveQuery) expected.search = '?probe=1';
+ assert.equal(location.href,expected.href,from);
+ }
+ for (const locale of locales) {
+ const prefix = locale === 'en' ? '' : `/${locale}`;
+ const html = await (await
fetch(origin+prefix+'/docs/intro/')).text();
+ assert(html.includes(locale === 'en' ? 'FixtureSQL documents'
: `FixtureSQL ${locale} documents`));
+ const $ = require('cheerio').load(html);
+ const urls =
$('script[src],link[rel="stylesheet"],img[src]').map((_,node) =>
$(node).attr('src') ?? $(node).attr('href')).get();
+ for (const resource of urls.filter(value =>
!value.startsWith('data:'))) {
+ const response = await fetch(new URL(resource,origin));
+ assert.equal(response.status,200,resource);
+ }
+ assert.equal((await
fetch(origin+prefix+'/not-a-page')).status,404);
+ }
+ }
+ let browser;
+ if (process.argv.includes('--browser')) {
+ const {chromium} = require('playwright');
+ browser = await chromium.launch({headless:true});
+ try {
+ for (const locale of locales) {
+ const page = await browser.newPage();
+ const errors = [];
+ page.on('pageerror',error => errors.push(error.message));
+ page.on('response',response => {if (response.status() >=
400) errors.push(`${response.status()} ${response.url()}`);});
+ const prefix = locale === 'en' ? '' : `/${locale}`;
+ await
page.goto(previewOrigin+prefix+'/',{waitUntil:'networkidle'});
+ await page.getByRole('button',{name:'Load
module'}).click();
+ await page.getByText('LAZY_OK').waitFor();
+ await page.getByRole('button',{name:'Search
module'}).click();
+ await page.getByText('SEARCH_OK_'+locale).waitFor();
+ await page.goto((apache?.origin ??
previewOrigin)+prefix+'/docs/intro.html/?probe=1#example',{waitUntil:'networkidle'});
+
assert(page.url().endsWith(prefix+'/docs/intro/?probe=1#example'));
+ const imageUrl = await page.locator('article
img').getAttribute('src');
+ assert.equal(hash(Buffer.from(await (await fetch(new
URL(imageUrl,previewOrigin))).arrayBuffer())),hash(image));
+ assert.deepEqual(errors,[],locale);
+ await page.close();
+ }
+ } finally {await browser.close();}
+ }
+ const result =
{baseline,...report.optimization,bundler:process.argv.includes('--webpack') ?
'webpack' : 'rspack',
+ apache:!!apache,browser:process.argv.includes('--browser')};
+
fs.writeFileSync(path.join(fixture,'report.json'),JSON.stringify(result,null,2)+'\n');
+ console.log(JSON.stringify(result,null,2));
+ } finally {
+ if (apache) await apache.close();
+ await new Promise(resolve => server.close(resolve));
+ }
+}
+if (require.main === module) main().catch(error =>
{console.error(error);process.exitCode=1;});
diff --git a/scripts/publish/verify.cjs b/scripts/publish/verify.cjs
new file mode 100644
index 00000000000..b414bd6fe3f
--- /dev/null
+++ b/scripts/publish/verify.cjs
@@ -0,0 +1,54 @@
+const fs = require('node:fs');
+const path = require('node:path');
+const assert = require('node:assert/strict');
+const {inventory} = require('./optimize.cjs');
+
+function verify(site) {
+ site = path.resolve(site);
+ const manifest =
JSON.parse(fs.readFileSync(path.join(site,'.publish/redirects.json'),'utf8'));
+ const redirects = new Map(manifest.redirects.map(([from,to]) =>
[decodeURIComponent(from),to]));
+ const files = inventory(site);
+ const available = new Set(files.map(item => item.relative));
+ const assetOrigin = require('../../config/asset-origin').getAssetOrigin();
+ const ownOrigins = new Set(['https://doris.apache.org',...(assetOrigin ?
[new URL(assetOrigin).origin] : [])]);
+ const resolve = value => {
+ let url = new URL(value,'https://doris.apache.org');
+ const key = decodeURIComponent(url.pathname);
+ const target = redirects.get(key) ?? redirects.get(key+'/');
+ if (target) url = new URL(target,url);
+ let relative = decodeURIComponent(url.pathname).replace(/^\//,'');
+ const prefix = manifest.assetPrefixes.find(prefix =>
url.pathname.startsWith(prefix));
+ if (prefix) relative =
'assets/'+decodeURIComponent(url.pathname.slice(prefix.length));
+ return available.has(relative) ||
available.has(path.posix.join(relative,'index.html'));
+ };
+ for (const [from,to] of manifest.redirects) {
+ if (to.startsWith('/')) assert(resolve(to), `Missing redirect
destination: ${from} -> ${to}`);
+ }
+ let references = 0;
+ const check = (value,source) => {
+ if (value.startsWith('data:')) return;
+ const url = new URL(value,`https://doris.apache.org/${source}`);
+ if (!ownOrigins.has(url.origin)) return;
+ if (!url.pathname.startsWith('/assets/')) return;
+ assert(resolve(url.href),`Missing compiled asset in ${source}:
${value}`);
+ references++;
+ };
+ for (const item of files) {
+ if (item.relative.endsWith('.html')) {
+ const content = fs.readFileSync(item.file,'utf8');
+ for (const tag of
content.matchAll(/<(?:script|link|img|source|video|audio)\b[^>]*>/g)) {
+ for (const attribute of
tag[0].matchAll(/\b(?:src|href)=(?:"([^"]*)"|'([^']*)'|([^\s>]+))/g)) {
+ check(attribute[1] ?? attribute[2] ??
attribute[3],item.relative);
+ }
+ }
+ } else if (item.relative.endsWith('.css')) {
+ for (const match of
fs.readFileSync(item.file,'utf8').matchAll(/url\(["']?([^\s"')]+)["']?\)/g))
check(match[1],item.relative);
+ }
+ }
+ return
{files:files.length,redirects:manifest.redirects.length,compiledAssetReferences:references};
+}
+
+if (require.main === module) {
+ console.log(JSON.stringify(verify(process.argv[2] ?? 'build'),null,2));
+}
+module.exports = {verify};
diff --git a/src/components/Icons/doris-logo-horizontal.tsx
b/src/components/Icons/doris-logo-horizontal.tsx
index 7cd81a205e0..0a25eb41415 100644
--- a/src/components/Icons/doris-logo-horizontal.tsx
+++ b/src/components/Icons/doris-logo-horizontal.tsx
@@ -1,35 +1,14 @@
import React from 'react';
+import useBaseUrl from '@docusaurus/useBaseUrl';
interface DorisLogoHorizontalProps {
className?: string;
}
-/**
- * Inline copy of static/images/brand-assets/doris-logo-horizontal-white.svg.
- *
- * The brand file has a 318x193 canvas with ~40 units of empty margin on every
- * side of the artwork, so an <img> box is a third wider than the visible logo.
- * The viewBox here is cropped to the artwork (x 40-278, y 55-138) so the box
- * hugs it, which is what lets the mobile navbar fit everything on one row.
- */
+// Preserve the original cropped artwork and aspect ratio while sharing its
+// paths across every static page instead of repeating them in the HTML.
export function DorisLogoHorizontal({ className }: DorisLogoHorizontalProps) {
- return (
- <svg
- className={className}
- viewBox="40 55 238 83"
- fill="#fff"
- xmlns="http://www.w3.org/2000/svg";
- aria-hidden="true"
- focusable="false"
- >
- <path
d="M77.24,68.09l-8.85-8.85c-2.35-2.35-5.54-3.63-8.64-3.63-2.88,0-5.76,1.07-8,3.2-2.24,2.24-3.52,5.12-3.52,8.32s1.17,6.08,3.41,8.32l16.1,16.1c.64.64,1.81.64,2.56,0l6.93-6.93c.43-.53,7.78-8.85,0-16.53Z"
/>
- <path
d="M91.85,82.27c-1.71-1.6-3.41-3.31-5.01-5.22l-.11-.11c0,.11-.11.32-.11.53-.64,3.84-2.45,7.25-5.44,10.24-9.38,9.28-18.87,18.87-28.04,28.04l-1.17,1.17c-2.13,2.13-3.31,4.37-3.63,6.61-.43,3.41.75,7.25,3.31,10.02,2.24,2.45,5.22,3.73,8.42,3.62,3.63.11,5.44-.43,8.1-2.98,10.66-10.45,21.32-21,29.43-29,3.84-3.84,4.69-10.02,2.03-14.29-2.24-3.31-5.01-5.97-7.78-8.64Z"
/>
- <path
d="M40.25,76.73v39.34c0,1.07.75,1.71,1.39,2.03.64.32,1.6.32,2.35-.43l19.83-19.83c.75-.75.75-2.03,0-2.88l-19.83-19.83c-.43-.43-1.07-.64-1.49-.64-.32,0-.64.11-.85.21-.64.32-1.39.96-1.39,2.03Z"
/>
- <path
d="M126.61,113.08v-33.37h9.92c10.87,0,17.59,6.4,17.59,16.63s-6.72,16.63-17.59,16.63h-9.92v.11ZM131.94,108.29h4.8c7.36,0,12.05-4.69,12.05-11.83s-4.69-11.83-12.05-11.83h-4.8v23.67Z"
/>
- <path
d="M177.89,113.62c-9.6,0-17.38-7.78-17.38-17.27s7.78-17.17,17.38-17.17,17.38,7.57,17.38,17.17c.11,9.7-7.57,17.27-17.38,17.27ZM177.89,84.08c-6.61,0-12.05,5.44-12.05,12.26s5.44,12.37,12.05,12.37,12.15-5.65,12.15-12.37-5.44-12.26-12.15-12.26Z"
/>
- <path d="M244.42,79.71h-5.44v33.37h5.44v-33.37Z" />
- <path
d="M264.57,113.62c-4.69,0-8.53-1.39-10.24-2.24v-4.9c2.35,1.07,6.08,2.35,10.24,2.35s7.14-1.92,7.14-4.8c0-3.2-3.52-4.58-7.14-5.86-5.76-2.13-10.13-4.16-10.13-9.7,0-6.82,6.18-9.28,11.52-9.28,4.05,0,7.25,1.07,8.74,1.6v4.8c-2.13-.85-5.33-1.81-8.85-1.81-4.05,0-5.97,2.13-5.97,4.26,0,2.88,3.09,3.94,6.72,5.22,6.29,2.24,11.09,4.48,11.09,10.45-.11,5.86-5.44,9.92-13.11,9.92Z"
/>
- <path
d="M219.9,99.54l2.03-.85c2.77-1.17,5.97-4.26,5.97-9.06,0-5.97-4.37-9.92-10.98-9.92h-12.69v33.37h5.33v-28.68h6.72c4.05,0,6.18,2.99,6.18,5.86s-2.13,5.86-6.18,5.86h-2.88c-1.28.21-.64,1.28-.64,1.28l10.77,15.57h5.97l-9.6-13.43Z"
/>
- </svg>
- );
+ const source =
useBaseUrl('/images/brand-assets/doris-logo-horizontal-mark.svg');
+ return <img className={className} src={source}
+ width="238" height="83" alt="" aria-hidden="true" />;
}
diff --git a/static/images/brand-assets/doris-logo-horizontal-mark.svg
b/static/images/brand-assets/doris-logo-horizontal-mark.svg
new file mode 100644
index 00000000000..4e8b18c19dd
--- /dev/null
+++ b/static/images/brand-assets/doris-logo-horizontal-mark.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg"; viewBox="40 55 238 83" width="238"
height="83" fill="#fff"><path
d="M77.24,68.09l-8.85-8.85c-2.35-2.35-5.54-3.63-8.64-3.63-2.88,0-5.76,1.07-8,3.2-2.24,2.24-3.52,5.12-3.52,8.32s1.17,6.08,3.41,8.32l16.1,16.1c.64.64,1.81.64,2.56,0l6.93-6.93c.43-.53,7.78-8.85,0-16.53Z"
/><path
d="M91.85,82.27c-1.71-1.6-3.41-3.31-5.01-5.22l-.11-.11c0,.11-.11.32-.11.53-.64,3.84-2.45,7.25-5.44,10.24-9.38,9.28-18.87,18.87-28.04,28.04l-1.17,1.17c-2.13,2.13-3.31,4.37-3.63,6.
[...]
diff --git a/static/js/brand-theme-bootstrap.js
b/static/js/brand-theme-bootstrap.js
new file mode 100644
index 00000000000..474fb296a1f
--- /dev/null
+++ b/static/js/brand-theme-bootstrap.js
@@ -0,0 +1,40 @@
+(function () {
+ var theme = 'doris';
+ try {
+ var storedTheme = localStorage.getItem('doris-brand-theme');
+ var themes = ['doris', 'golden', 'blue', 'read', 'yellow-blue',
'purple', 'yellow-black', 'sky'];
+ if (themes.indexOf(storedTheme) !== -1) theme = storedTheme;
+ } catch (error) {}
+ document.documentElement.setAttribute('data-brand-theme', theme);
+
+ var kapaSelector =
'script[data-website-id="a5fb90df-217a-4097-95c0-80490220314b"]';
+ var kapaColors = {
+ doris: '#11A679',
+ golden: '#7B2CBF',
+ blue: '#2C2C34',
+ read: '#121212',
+ 'yellow-blue': '#0066FF',
+ purple: '#7255A5',
+ 'yellow-black': '#000000',
+ sky: '#3778B0'
+ };
+ var kapaColor = kapaColors[theme];
+ var observer;
+ function syncKapaScript(node) {
+ if (node.nodeType !== 1) return false;
+ var script = node.matches && node.matches(kapaSelector)
+ ? node
+ : node.querySelector && node.querySelector(kapaSelector);
+ if (!script) return false;
+ script.setAttribute('data-project-color', kapaColor);
+ if (observer) observer.disconnect();
+ return true;
+ }
+ observer = new MutationObserver(function (mutations) {
+ mutations.some(function (mutation) {
+ return Array.prototype.some.call(mutation.addedNodes,
syncKapaScript);
+ });
+ });
+ observer.observe(document.documentElement, { childList: true, subtree:
true });
+ syncKapaScript(document.documentElement);
+}());
diff --git a/static/js/course-rail-bootstrap.js
b/static/js/course-rail-bootstrap.js
new file mode 100644
index 00000000000..cad0a79ee09
--- /dev/null
+++ b/static/js/course-rail-bootstrap.js
@@ -0,0 +1,7 @@
+(function () {
+ try {
+ if (localStorage.getItem('doris-101-rail-collapsed') === 'true') {
+ document.documentElement.setAttribute('data-course-rail',
'collapsed');
+ }
+ } catch (error) {}
+}());
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
