Hey everyone,
Last month, I mentioned that I was going to start working on automated
website builds for Calcite and its sub-projects. I have completed most
of the work but am current stuck on how to trigger them.
The commits are currently in this branch:
https://github.com/apache/calcite/tree/test-site
The jenkins job is here: https://builds.apache.org/job/calcite-site/
First off, here are my proposed changes to the way websites are built:
1. Force the Jekyll build environment/docker image to use UTC as its
default timezone:
https://github.com/apache/calcite/blob/test-site/site/_config.yml#L58
and
https://github.com/apache/calcite/blob/test-site/site/docker-compose.yml#L33
This allows us to have a definite reference point to calculate the dates
for news items. I think UTC is a pretty "neutral" timezone compared to
other ones and it's often much easier for people to calculate the date
mentally as we tend to know how many hours ahead or behind UTC we are in.
2. Allow publication of future
posts:https://github.com/apache/calcite/blob/test-site/site/_config.yml#L56
When releasing avatica and avatica-go, I often find it annoying to have
to wait for a certain time before building and publishing the release
item (for it to show up) due to the release date. I think if the news
item is published a couple of hours earlier, not much harm is done,
considering that if we have automated website builds in place, it would
be quite annoying to have to push another commit and have it build to
show up.
3. Add a site branch for calcite-avatica and calcite-avatica-go. The
avatica and avatica-go sites don't tend to get updated all that much. We
usually just push it, along with the news item after a release. However,
there may be some commits that make changes to the documentation on the
site as part of a PR and we do not want to build the site from master.
If we build when the repository is tagged, the news item won't be
published as it's committed after the tag's commit. Therefore, we should
have a site branch (like calcite) for those 2 repositories where
everything on site is build immediately, and after a release, the RM
should even the master and site branches.
4. Build triggers:
- For Javadocs (Calcite and Avatica) trigger and build when a final
version is tagged.
- Build the site branch (Calcite, Avatica and Avatica Go) every time a
new commit is added.
5. For the calcite-site repository, we need to push the changes to the
`asf-site` branch and ask infra to set gitpubsub to that branch. The
reason for this is that the git credentials on the gitwebsites build
nodes can only push to a branch called `asf-site`.
Please let me know what you guys think of these proposals.
Problem:
So, the build works correctly and I have verified this using the Jenkins
job. However, when it comes to triggering the jobs, it seems to not be
working correctly. My memory is a bit hazy as I last worked on this a
few weeks ago, but if I am just triggering the build on a push to the
`test-site` branch, everything works correctly. However, once I try to
get it to trigger on a tag, sometimes it doesn't trigger and sometimes,
it just skips all the builds in the Jenkins pipeline.
I think Jenkins is a much more complex beast than I've anticipated and I
currently don't have enough bandwidth to dig into its internals. Could
someone who knows more about Jenkins find some time to look into it?
Another solution might be to use an external CI such as Travis (which is
easier to configure), but I am not sure how happy INFRA would be with
putting the git credential as a secret on those services. There is
currently some general discussion for external CI services on the builds
list here:
https://lists.apache.org/thread.html/af52e2a3e865c01596d46374e8b294f2740587dbd59d85e132429b6c@%3Cbuilds.apache.org%3E
Francis
