Re: [DISCUSS] Best practice for synchronizing master and site branches

Thanks for the feedback, everyone. I'll add comments and improve the structure to make things more clear and readable. I will also land similar changes in calcite-avatica and calcite-avatica-go soon so that we can get the first iteration out. Francis On 31/03/2022 6:22 am, Julian Hyde wrote:

Re: [DISCUSS] Best practice for synchronizing master and site branches

I had a quick look and it looks like clean well-thought-out code. I couldn’t figure what it was doing at a high level (e.g. what the generated URLs would look like), so I think some high-level comments would help. +1 Thanks for your excellent work, Francis. Julian > On Mar 30, 2022, at 11:54

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hi Francis, I went over the workflows and rules and everything looks good to me. Great work! I'm +1 on merging this to master and I am OK with your suggestions about Avatica. Thanks a lot for moving this forward. It will certainly save us a lot of time in the future. When this goes in we need t

Re: [DISCUSS] Best practice for synchronizing master and site branches

Forgot to mention in my last message, but I am now implementing the automation for calcite-avatica and calcite-avatica-go For those 2 repos, we never used a site branch as we usually push the site after a release. If there are any small updates to the site that occur after the release, we just

Re: [DISCUSS] Best practice for synchronizing master and site branches

I have implemented automatic site builds for Calcite in a test repo. See: - https://github.com/F21/calcite-test/blob/master/.github/workflows/publish-non-release-website-updates.yml - https://github.com/F21/calcite-test/blob/master/.github/workflows/publish-website-on-release.yml For the auto

Re: [DISCUSS] Best practice for synchronizing master and site branches

Infra has added the Github Actions secret to push to the calcite-site repo [1]. Let's keep the process we have at the moment, but automate it. I'll start by making test repos mirroring the 4 calcite-* repos on my account to test the automation. In the meantime can members of the community le

Re: [DISCUSS] Best practice for synchronizing master and site branches

I have never needed or wanted a versioned Javadoc URL for Calcite. Our APIs tend to grow over time. The only requirement I see is that we don’t pollute the javadoc/doc of the latest released version with things that are not yet released. Which would lead to two versions: latest release and head

Re: [DISCUSS] Best practice for synchronizing master and site branches

I think it is a good idea to provide versioned JavaDocs. However, even if we only provide the JavaDoc of the latest release, there is no need to maintain two branches (IMHO), because the processes of updating the website and JavaDoc are relatively separate processes (according to [1]). With a sing

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hello everyone, I totally agree on automating the website publication and having a single branch, the less we do manually, the lower the chances to mess something up. I am also in favour of versioned docs in the website, it's confusing to land on updated pages from an older context like a message

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hey Julian, All very good points. I can definitely see the utility of the javadocs. The analogue in Go would be godoc, with the difference being that the godoc server automatically crawls the code across all versions to generate the documentation. As an example, see the godoc for protobuf [1

Re: [DISCUSS] Best practice for synchronizing master and site branches

To respond to Stamatis’ query: I do use the generated javadoc. For example, if I am talking about a connection property I might email the following link: https://calcite.apache.org/javadocAggregate/org/apache/calcite/config/CalciteConnectionProperty.html#FORCE_DECORRELATE

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hi all, Thanks for the fruitful discussion. It seems there is not a "single formula" that is safe for all scenarios. In addition, it requires some effort to recognize the particular scenario we are in (commits out of order, missing commits, etc.). Sometimes it is not reliable to simply check the

Re: [DISCUSS] Best practice for synchronizing master and site branches

If all version specific documentation is siloed into their own respective folders for each version, then this will be much easier to automate as we can just simply build and publish the site on every push to master. Each version would be in each folder, with the unreleased version being in th

Re: [DISCUSS] Best practice for synchronizing master and site branches

Having multiple APIs versions in the website has been discussed here [1]. Since this work of automation is important and has been postponed many times in the past I think it is important to get something simple to begin with and add "new features" like versioned documentation later on and if there

Re: [DISCUSS] Best practice for synchronizing master and site branches

Would it be clearer if we had different API versions on the site? We could have one API link per Calcite version (or at least for the latest X versions) + an API link of the current master head (that could be updated automatically). I think this "multiple API" idea has been already discussed in the

Re: [DISCUSS] Best practice for synchronizing master and site branches

It looks like Infra should be able to give us a token to push to calcite-site from our other calcite-* repos using Github actions [1]. If we can have some consensus regarding whether to keep the site branch and maintain the current process, or to remove it and just publish from master, I can s

Re: [DISCUSS] Best practice for synchronizing master and site branches

It would be great if we manage to wrap up CALCITE-3129 and have an automated build for the website. The thing that complicates the procedure in general (automated or not) is the fact that we don't want to publish API related changes on the web before they are officially released. I understand the

Re: [DISCUSS] Best practice for synchronizing master and site branches

Ideally, I would like to see that the site builds are automated by CI, we still have CALCITE-3129 [1] open. My thinking is that if we automate the site building and deployment process, we can use the following heuristics: - Build the site completely and deploy when a final release tag is pushe

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hello, Thanks for starting this discussion Liya. It is important to find which parts of the process are unclear and improve them if possible. The current procedure for updating the website remains unchanged and it is documented here: https://github.com/apache/calcite/blob/a6a1e2cef332893fd9028609

Re: [DISCUSS] Best practice for synchronizing master and site branches

Does anyone know (or could find out) the SHA of the master and site branches at the time that Fan attempted to move the site changes over? If so, we could recreate the same environment, and figure out a set of git commands that would have worked then and will work for the next release manager. This

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hi Francis, Thanks for your feedback. It seems we should choose option 2. In addition, it seems less risky to run "git push --force" commands in the site branch. Best, Liya Fan Francis Chuang 于2022年3月25日周五 12:14写道： > > Hi Liya, > > Thanks for bringing this up. We have always done the following

Re: [DISCUSS] Best practice for synchronizing master and site branches

Hi Liya, Thanks for bringing this up. We have always done the following when committing: 1. Always commit to master. 2. If we need to publish the change to the site now (for example, new committer or announcement), cherry-pick the change into the site branch and publish it. 3. After a release

[DISCUSS] Best practice for synchronizing master and site branches

Hi all, As part of the release process, we need to synchronize the master and site branches (Please see https://calcite.apache.org/docs/howto.html#making-a-release-candidate). Usually, the site is behind the master branch by some commits. If the existing commits in the site branch are in the same