I have a POC for a reliable way to locally build small parts of the (Antora-built) website, e.g. a subproject.
Prerequisites: 1. camel-website must be cloned next to the subproject: e.g. if the “main” directory is camel-projects, you’ll have camel-projects/camel-website and camel-projects/camel-quarkus. Note: it would be possible to have camel-website under the subproject, but that would make working on more than one subproject implausible. 2. You’re using linux, macOS, or another unix-like OS…. there’s a shell script. What to do: 1. Initially build the camel ui. This is easy on linux systems and difficult on macOS; you have to run yarn to recompile some dependencies. It may be easier to build in docker. 2. In the subproject, initially run `./loca-build.sh full` once to perform a full Antora build that includes your local subproject state, to generate the ‘site-manifest’ needed for the partial build. 3. After this, run `./local-build.sh` to continuously update the full build with changes in the subproject. This starts a local web server which is slightly more capable than looking at the file urls. This does an initial partial build, and then watches for file changes. The whole site is viewable. Limitations: 1. At the moment, xref checks from the subproject to the rest of the site are validated correctly, but xrefs into the subproject from the rest of the site are not. I’m hoping to extend the site-manifest to fix this. 2. You have to do a full build locally once to construct the site manifest and the local content from the parts of the site outside the subproject. This takes about 7 minutes on my laptop. If there are significant changes to the website you need to redo this local build, and there’s no way to detect when this is needed. 3. You have to build the camel ui once which is difficult on non-linux systems. This could easily be fixed by checking in the built ui bundle whenever it changes. Possible changes/future work: 1. The site-manifest can be built at minimal cost during the official website build, and served from the website. It’s about 2.5 MB, so I don’t think it’s a significant drain on resources. Doing just this would eliminate the need for step (2), building the full site locally, and you’d still have good xref checks, but with just this step you’d get a site where only the subproject pages are local, and links out of the subproject go to the main site; links into the subproject would stay on the main site. I don’t like this much, but perhaps it’s completely adequate for most site development. 2. In addition to (1), the zip-publisher could be added to the build, to zip up the whole Antora part of the site; this could be downloaded instead of step (2). This would be pretty easy to set up, but the resulting zip is about 110MB. I don’t know what infra would think of serving this from the root of the website. This would completely eliminate the need for step (2) unless you wanted to do a full local build with your changes, perhaps to check xrefs into the subproject or to check other unusual interactions. 3. Some of this may be adaptable to building PRs that change the site content, with just a subproject PR and not an additional camel-website PR. This depends on how much information can be sent from one GitHub action to another in a different repository. Current state: I’ve set this up for camel-quarkus main in its current state and camel-kamelets main showing how I propose to generate the kamelet pages on the fly with Antora. I can set this up for other subprojects on request. How it works: There are three Antora extensions, some per-subproject playbook additions, and a shell script. @djencks/antora-source-map uses configuration to modify the playbook source/branch info, for each configured source/branch replacing the apache GitHub repo with the local clone and branch. (this is what would be used in "future (3)’) @djencks/antora-site-manifest, which is an adaptation and modification of something Dan Allen developed, generates the site-manifest in the full build and imports it in the partial build. @djencks/antora-source-watch restricts which files are loaded into the content catalog (rather than being represented by the site-manifest entry) and sets up a file watcher to detect changes and a web server to show the site; on a file change it rebuilds the site. browser-watch refreshes any pages you may be looking at from the site. camel-website has a playbook addition that configures the site manifest generation on a full build. This could be added to the playbook if we decided to always generate the site-manifest (future (1)). There are also a couple of yarn scripts to do full and partial antora builds. Each subproject has two playbook additions: - source-map.yml to configure the source-map extension - source-watch.yml to configure the source-watch extension and a script that concatenates the additions to the main Antora playbook to result in local playbooks for full and partial builds, and runs the builds. Where is it?!?!? Camel-website issue: https://github.com/apache/camel-website/issues/720 <https://github.com/apache/camel-website/issues/720> camel-website PR https://github.com/apache/camel-website/pull/721 <https://github.com/apache/camel-website/pull/721> camel-quarkus PR https://github.com/apache/camel-quarkus/pull/3385 <https://github.com/apache/camel-quarkus/pull/3385> camel-kamelets PR https://github.com/apache/camel-kamelets/pull/630 <https://github.com/apache/camel-kamelets/pull/630> I think you can try this out by making sure your clones are next to one another as in prerequisite (1), pulling down my branches, and following the instructions. Comments? David Jencks
