This is an automated email from the ASF dual-hosted git repository.
raulcd pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-js.git
The following commit(s) were added to refs/heads/main by this push:
new 2e0ef8b chore: Publish docs (#166)
2e0ef8b is described below
commit 2e0ef8bba12dc60ab462dcfb80ce85e7ddb3ad6f
Author: Sutou Kouhei <[email protected]>
AuthorDate: Thu Jun 19 20:10:48 2025 +0900
chore: Publish docs (#166)
## What's Changed
Build docs by CI and:
Push an RC tag:
* Upload the built docs to GitHub Releases
Push a release tag:
* Publish the uploaded docs to
`https://arrow.apache.org/js/{current,${VERSION}}` via the asf-site
branch
Push to main in apache/arrow-js:
* Publish the uploaded docs to https://arrow.apache.org/js/main via the
asf-site branch
Push to non main in apache/arrow-js:
* Do nothing
Push to foks:
* Upload the built docs to GitHub Pages via the gh-pages branch for
preview
* Example: https://kou.github.io/arrow-js/
* Developers need to enable GitHub Pages explicitly at
https://github.com/kou/arrow-js/settings/pages
Closes #10.
---------
Co-authored-by: Raúl Cumplido <[email protected]>
---
.github/workflows/rc.yaml | 70 ++++++++++++++++++++++++++++
.github/workflows/release.yaml | 28 +++++++++++
DEVELOP.md | 31 +++++++++++--
ci/scripts/update_docs.sh | 101 ++++++++++++++++++++++++++++++++++++++++
dev/release/README.md | 14 +++---
dev/release/docs.md | 102 +++++++++++++++++++++++++++++++++++++++++
dev/release/release_rc.sh | 7 ++-
7 files changed, 337 insertions(+), 16 deletions(-)
diff --git a/.github/workflows/rc.yaml b/.github/workflows/rc.yaml
index 2c626bf..62bbc82 100644
--- a/.github/workflows/rc.yaml
+++ b/.github/workflows/rc.yaml
@@ -87,6 +87,75 @@ jobs:
run: |
dev/release/run_rat.sh "${TAR_GZ}"
+ docs:
+ name: Documentation
+ needs: target
+ runs-on: ubuntu-latest
+ timeout-minutes: 5
+ permissions:
+ contents: write
+ env:
+ RC: ${{ needs.target.outputs.rc }}
+ VERSION: ${{ needs.target.outputs.version }}
+ steps:
+ - name: Checkout
+ uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 #
v4.2.2
+ - uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 #
v4.4.0
+ with:
+ cache: yarn
+ node-version: 20
+ - name: Install dependencies
+ run: |
+ yarn install
+ - name: Build
+ run: |
+ yarn doc
+ - name: Package
+ run: |
+ id="apache-arrow-js-docs-${VERSION}"
+ tar_gz="${id}.tar.gz"
+ mv doc "${id}"
+ tar czf "${tar_gz}" "${id}"
+ sha256sum "${tar_gz}" > "${tar_gz}.sha256"
+ sha512sum "${tar_gz}" > "${tar_gz}.sha512"
+ - name: Upload
+ uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02
# v4.6.2
+ with:
+ name: release-docs
+ path: |
+ apache-arrow-js-docs-*.tar.gz*
+ - name: Prepare for publish
+ run: |
+ need_publish=no
+ branch=
+ if [ "${GITHUB_EVENT_NAME}" = "push" ]; then
+ if [ "${GITHUB_REPOSITORY}" = "apache/arrow-js" ]; then
+ branch=asf-site
+ if [ "${GITHUB_REF_NAME}" = "main" ]; then
+ need_publish=yes
+ fi
+ else
+ branch=gh-pages
+ need_publish=yes
+ fi
+ fi
+ if [ "${need_publish}" = "yes" ]; then
+ ci/scripts/update_docs.sh "${VERSION}" "${branch}"
+ fi
+ echo "NEED_PUBLISH=${need_publish}" >> "${GITHUB_ENV}"
+ - name: Publish
+ if: env.NEED_PUBLISH == 'yes'
+ run: |
+ cd site
+ cp -a ../.asf.yaml ./
+ git add .asf.yaml
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+ if [ "$(git diff --cached)" != "" ]; then
+ git commit -m "Update ${GITHUB_SHA}"
+ git push origin "$(git branch --show-current)"
+ fi
+
packages:
name: Packages
runs-on: ubuntu-latest
@@ -153,6 +222,7 @@ jobs:
upload:
name: Upload
needs:
+ - docs
- target
- verify
runs-on: ubuntu-latest
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
index 306d007..8d53f02 100644
--- a/.github/workflows/release.yaml
+++ b/.github/workflows/release.yaml
@@ -56,3 +56,31 @@ jobs:
--title "Apache Arrow JS ${version}" \
--verify-tag \
dists/*
+ - name: Checkout asf-site
+ uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 #
v4.2.2
+ with:
+ ref: asf-site
+ path: site
+ - name: Update documentation
+ run: |
+ version=${GITHUB_REF_NAME#v}
+ tar_gz=${PWD}/dists/apache-arrow-js-docs-${version}.tar.gz
+
+ pushd site
+
+ rm -rf current
+ mkdir -p current
+ pushd current
+ tar xf "${tar_gz}" --strip-components=1
+ popd
+ git add current
+
+ rm -rf "${version}"
+ cp -a current "${version}"
+ git add "${version}"
+
+ git config user.name "github-actions[bot]"
+ git config user.email "github-actions[bot]@users.noreply.github.com"
+
+ git commit -m "Update docs for ${version}"
+ git push origin "$(git branch --show-current)"
diff --git a/DEVELOP.md b/DEVELOP.md
index 324414b..52e9bfb 100644
--- a/DEVELOP.md
+++ b/DEVELOP.md
@@ -17,7 +17,9 @@
under the License.
-->
-# Getting Involved
+# How to develop
+
+## Getting Involved
Even if you do not plan to contribute to Apache Arrow itself or Arrow
integrations in other projects, we'd be happy to have you involved:
@@ -40,7 +42,7 @@ If you’d like to report a bug but don’t have time to fix it,
you can still p
it on GitHub issues, or email the mailing list
[[email protected]](http://mail-archives.apache.org/mod_mbox/arrow-dev/)
-# The package.json scripts
+## The package.json scripts
We use [yarn](https://yarnpkg.com/) to install dependencies and run scrips.
@@ -70,19 +72,19 @@ To run tests directly on the sources without bundling, use
the `src` target (e.g
Compiles the documentation with [Typedoc](https://typedoc.org/). Use `yarn doc
--watch` to automatically rebuild when the docs change.
-# Running the Performance Benchmarks
+## Running the Performance Benchmarks
You can run the benchmarks with `yarn perf`. To print the results to stderr as
JSON, add the `--json` flag (e.g. `yarn perf --json 2> perf.json`).
You can change the target you want to test by changing the imports in
`perf/index.ts`. Note that you need to compile the bundles with `yarn build`
before you can import them.
-# Testing Bundling
+## Testing Bundling
The bundles use `apache-arrow` so make sure to build it with `yarn build -t
apache-arrow`. To bundle with a variety of bundlers, run `yarn test:bundle` or
`yarn gulp bundle`.
Run `yarn gulp bundle:webpack:analyze` to open [Webpack Bundle
Analyzer](https://github.com/webpack-contrib/webpack-bundle-analyzer).
-# Updating the Arrow format flatbuffers generated code
+## Updating the Arrow format flatbuffers generated code
1. Once generated, the flatbuffers format code needs to be adjusted for our
build scripts (assumes `gnu-sed`):
@@ -110,6 +112,25 @@ Run `yarn gulp bundle:webpack:analyze` to open [Webpack
Bundle Analyzer](https:/
4. Execute `yarn lint` from the `js` directory to fix the linting errors
+## How to preview documentation on your fork repository
+
+Our GitHub Actions workflows will create the `gh-pages` branch on your
+fork repository automatically. It's for previewing documentation with
+your changes in a branch.
+
+You need to enable GitHub Pages on your fork repository at
+`https://github.com/${YOUR_GITHUB_ID}/arrow-js/settings/pages`
+manually. Choose the `gh-pages` branch in the "Build and deploy" ->
+"Branch" configuration item and press the "Save" button. We can
+preview documentation by GitHub Pages for your fork repository:
+`https://${YOUR_GITHUB_ID}.github.io/arrow-js/`
+
+Example: https://kou.github.io/arrow-js/
+
+The `gh-pages` branch keeps previews of all branches. We recommend
+that you delete needless branches and merged branches from your fork.
+
+
[1]: mailto:[email protected]
[2]: https://github.com/apache/arrow/tree/main/format
[3]: https://github.com/apache/arrow-js/issues
diff --git a/ci/scripts/update_docs.sh b/ci/scripts/update_docs.sh
new file mode 100755
index 0000000..cc37d21
--- /dev/null
+++ b/ci/scripts/update_docs.sh
@@ -0,0 +1,101 @@
+#!/usr/bin/env bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -exu
+
+version="${1}"
+target_branch="${2}"
+
+html_escape() {
+ # & -> & must be the first substitution
+ sed -e "s/&/&/g" \
+ -e "s/</</g" \
+ -e "s/>/>/g" \
+ -e "s/\"/"/g" \
+ -e "s/'/'/g"
+}
+
+if ! git fetch origin "${target_branch}"; then
+ git worktree add --orphan -b "${target_branch}" site
+else
+ git worktree add site "origin/${target_branch}"
+fi
+
+tar_gz="${PWD}/apache-arrow-js-docs-${version}.tar.gz"
+
+extract_docs() {
+ local destination="${1}"
+
+ rm -rf "${destination}"
+ mkdir -p "${destination}"
+ pushd "${destination}"
+ tar xf "${tar_gz}" --strip-components=1
+ popd
+ git add "${destination}"
+}
+
+pushd site
+if [ "${target_branch}" = "asf-site" ]; then
+ # Update https://arrow.apache.org/js/main/
+ extract_docs main
+
+ # Create .htaccess
+ cat >.htaccess <<HTACCESS
+RedirectMatch "^/js/$" "/js/current/"
+HTACCESS
+ git add .htaccess
+else
+ # Remove data for nonexistent branches
+ for branch in *; do
+ if [ ! -d "${branch}" ]; then
+ continue
+ fi
+ if ! git fetch origin "${branch}"; then
+ git rm "${branch}"
+ fi
+ done
+
+ # Update the pushed branch
+ extract_docs "${GITHUB_REF_NAME}"
+
+ # Create index.html
+ {
+ echo "<!DOCTYPE html>"
+ echo "<html>"
+ echo " <head>"
+ echo " <title>Apache Arrow JS documents</title>"
+ echo " </head>"
+ echo " <body>"
+ echo " <ul>"
+ for branch in *; do
+ if [ ! -d "${branch}" ]; then
+ continue
+ fi
+ escaped_branch="$(echo "${branch}" | html_escape)"
+ echo " <li>"
+ echo " <a href=\"${escaped_branch}/\">${escaped_branch}</a>"
+ echo " </li>"
+ done
+ echo " </ul>"
+ echo " </body>"
+ echo "</html>"
+ } >index.html
+ git add index.html
+fi
+popd
diff --git a/dev/release/README.md b/dev/release/README.md
index 469873d..6714c21 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -1,4 +1,4 @@
-<!---
+<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
@@ -28,7 +28,7 @@
5. Announce the new release on the mailing list (detailed later)
6. Announce the new release on social media (detailed later)
-### Prepare release environment
+## Prepare release environment
This step is needed only when you act as a release manager the first time.
@@ -79,7 +79,7 @@ $ head KEYS
$ svn ci KEYS
```
-### Bump version for new release
+## Bump version for new release
Open a PR that bumps version for new release. We must follow [Semantic
Versioning](https://semver.org/). For example, we must bump major
@@ -87,7 +87,7 @@ version when we have any incompatible changes.
You can proceed to the next step once we merge the opened PR.
-### Prepare RC and vote
+## Prepare RC and vote
You can use `dev/release/release_rc.sh`.
@@ -116,7 +116,7 @@ $ dev/release/release_rc.sh 1
The argument of `release_rc.sh` is the RC number. If RC1 has a
problem, we'll increment the RC number such as RC2, RC3 and so on.
-### Publish
+## Publish
We need to do the followings to publish a new release:
@@ -140,7 +140,7 @@ $ dev/release/release.sh 1
Add the release to ASF's report database via [Apache Committee Report
Helper](https://reporter.apache.org/addrelease.html?arrow).
-### Announce the new release on the mailing list
+## Announce the new release on the mailing list
Send an email to "[email protected]" from your Apache email, CC'ing
[email protected]/[email protected]. See an [example
@@ -185,7 +185,7 @@ Regards,
The Apache Arrow community.
```
-### Announce the new release on social media
+## Announce the new release on social media
Make a post on our [BlueSky](https://bsky.app/profile/arrow.apache.org) and
[LinkedIn](https://www.linkedin.com/company/apache-arrow/) accounts. (Ask
diff --git a/dev/release/docs.md b/dev/release/docs.md
new file mode 100644
index 0000000..9d8db6b
--- /dev/null
+++ b/dev/release/docs.md
@@ -0,0 +1,102 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# How to publish docs
+
+## Overview
+
+We publish our official docs to the following locations:
+
+* https://arrow.apache.org/js/current/ : For the current release
+* https://arrow.apache.org/js/${VERSION} : For the specified version such as
`21.0.0`
+* https://arrow.apache.org/js/main/ : For the `main` branch
+
+We also publish docs for fork repositories for preview. It uses the
+following locations:
+
+* https://${GITHUB_ID}.github.io/arrow-js/ : List all available docs
+* https://${GITHUB_ID}.github.io/arrow-js/${BRANCH}/ : For the `${BRANCH}`
branch
+
+All of them are automated. This documentation describes how it works.
+
+### apache/arrow-js
+
+apache/arrow-js uses the ASF provided Apache to host generated
+documentation. In general, the `asf-site` branch is used for it. See
+the `publish.whoami` configuration in the top-level `.asf.yaml`.
+
+Note that we don't need to touch the `asf-site` branch manually. It's
+completely maintained automatically. If the `asf-site` branch doesn't
+exist, it's created automatically.
+
+If we update the `asf-site` branch, the `asf-site` branch contents
+will be published to https://arrow.apache.org/js/ automatically.
+
+https://arrow.apache.org/js/ is always redirected to
+https://arrow.apache.org/js/current/ that provides the current release
+documentation. It's implemented by `.htaccess`. `.htaccess` is
+generated by `ci/scripts/update_docs.sh`.
+
+The `asf-site` branch is updated when:
+
+* We push a release tag
+* We push a commit to the `main` branch (We merge a PR to the `main` branch)
+
+Note that the `asf-site` branch isn't updated when we push an RC
+tag. We just build documentation and upload to a GitHub Release when
+we push an RC tag. We add the built documentation to the `asf-site`
+branch when we push a release tag. In this case, `current/` and
+`${VERSION}/` directories in the `asf-site` branch are updated.
+
+We build documentation and add the built documentation to the
+`asf-site` when we push a commit to the `main` branch. In this case,
+`main/` directory in the `asf-site` branch is updated.
+
+See also `ci/scripts/update_docs.sh`.
+
+### Fork repositories
+
+Fork repositories use GitHub Pages to host generated
+documentation. The `gh-pages` branch is used for it.
+
+Note that we don't need to touch the `gh-pages` branch manually. It's
+completely maintained automatically. If the `gh-pages` branch doesn't
+exist, it's created automatically. If your fork repository's size
+becomes bigger by the `gh-pages` branch, you can delete the `gh-pages`
+branch. Because it has only temporary data and can be generated
+automatically.
+
+You need to enable GitHub Pages on your fork repository. See
+[DEVELOP.md](../../DEVELOP.md#how-to-preview-documentation-on-your-fork-repository)
how to enable GitHub Pages on your fork repository.
+
+If we update the `gh-pages` branch, the `gh-page` branch contents will
+be published to https://${GITHUB_ID}.github.io/arrow-js/
+automatically.
+
+https://${GITHUB_ID}.github.io/arrow-js/ shows all available
+previews. https://${GITHUB_ID}.github.io/arrow-js/${BRANCH} has a
+preview of the `${BRANCH}` branch. If the `${BRANCH}` branch is
+deleted, the `${BRANCH}` directory is also deleted from the `gh-pages`
+branch when you push a commit to your fork repository.
+
+The `gh-pages` branch is updated when you push a commit to your fork
+repository. Our workflow generates a preview for the pushed branch and
+deletes previews of nonexistent branches.
+
+See also `ci/scripts/update_docs.sh`.
diff --git a/dev/release/release_rc.sh b/dev/release/release_rc.sh
index c191fda..36458f0 100755
--- a/dev/release/release_rc.sh
+++ b/dev/release/release_rc.sh
@@ -78,12 +78,11 @@ rc_hash="$(git rev-list --max-count=1 "${rc_tag}")"
artifacts_dir="apache-arrow-js-${version}-rc${rc}"
signed_artifacts_dir="${artifacts_dir}-signed"
+git_origin_url="$(git remote get-url origin)"
+repository="${git_origin_url#*github.com?}"
+repository="${repository%.git}"
if [ "${RELEASE_SIGN}" -gt 0 ]; then
- git_origin_url="$(git remote get-url origin)"
- repository="${git_origin_url#*github.com?}"
- repository="${repository%.git}"
-
echo "Looking for GitHub Actions workflow on ${repository}:${rc_tag}"
run_id=""
while true; do
