Re: Updating the Website
Hello, The process for updating the website is still a bit involved so people can easily get it wrong. I am reviving this thread to refresh people's memories around this topic. I also raised a PR [1] as an attempt to improve the respective instructions. Feedback is welcome, especially from people who haven't touched the website so far. Best, Stamatis [1] https://github.com/apache/calcite/pull/2708 On Thu, Dec 12, 2019 at 9:08 AM Francis Chuang wrote: > My plan is to get automated site builds up and running first, which > should get rid of the most difficult/troublesome steps for updating the > site. > > We can then evolve/experiment with the site to improve the process further. > > On 12/12/2019 6:28 pm, Stamatis Zampetakis wrote: > > I guess it will require some effort to setup and automate the process for > > supporting multiple versions but afterwards it may be easier to maintain. > > If the only thing that a committer has to do to update the site is > > committing to the master then there is not even need for a particular > > workflow. > > > > On Mon, Dec 9, 2019 at 10:31 PM Julian Hyde wrote: > > > >> We might be inventing requirements here, in order to justify a “cool” > >> technical change. > >> > >> I don’t think there is a strong requirement for multiple versions of the > >> site. (Sure, it would be nice.) > >> > >> This thread started with Stamatis pointing out that it was complicated > to > >> update the site. If we support multiple versions, will this actually > make > >> things less complicated? > >> > >> Julian > >> > >> > >> > >>> On Dec 9, 2019, at 1:23 PM, Stamatis Zampetakis > >> wrote: > >>> > >>> In the short term we should try to do our best to follow the existing > >>> workflow. > >>> > >>> In the medium term we shall hope that things will be easier with the > >>> automated build of the website. > >>> > >>> In the longer term, I would really prefer to migrate towards a solution > >>> like the one proposed by Vladimir. > >>> As I also mentioned in a previous email, there are many projects who > >>> publish multiple versions of the doc and I find this very helpful. > >>> People usually wait some time before updating their libraries to the > >> latest > >>> release; in this and other cases it is helpful to have a couple > versions > >> of > >>> the doc available online. > >>> > >>> > >>> On Sun, Dec 8, 2019 at 11:02 PM Vladimir Sitnikov < > >>> sitnikov.vladi...@gmail.com> wrote: > >>> > Francis>There are also links to Avatica docs in > Francis>the side bar and it would be a bit strange to have them always > point to > Francis>the master version of Avatica. > > gradle.properties references the Avatica version, so we could print > the > appropriate links. > > Michael>that need to be made that are independent of a particular > >> release > Michael>(e.g. adding a commiter)? > Michael>Would I go back and edit the previous > Michael>release branch? > > No. You update committers on a master branch > > Michael>Do we somehow label parts of the site as being > Michael>release-independent? > > It makes little sense to discuss. The answer will be obvious once > >> someone > tries. > > Michael>Even if this is the case, consider when we might > Michael>have to correct documentation errors from a revious release > > The current ASF rule is to have rel/... tag for each release. > That is the site build script could use rel/vX.Y tags to get "released > versions". > > Then there are at least two strategies. > a) If we want to update documentation for calcite-1.10.0, then we > could > release calcite-v1.10.1. > b) If a "silent" update is required (e.g. fix typo), then we could > >> invent > "support/vX.Y" branches, and commit the fix to that branch. > > Note: the current release process does not require a "release branch". > The build script does NOT create new commits to the source repository. > However, we could create one on-demand (e.g. in case we really need to > patch the old site version or back-port a fix) > > Vladimir > > >> > >> > > >
Re: Updating the Website
My plan is to get automated site builds up and running first, which should get rid of the most difficult/troublesome steps for updating the site. We can then evolve/experiment with the site to improve the process further. On 12/12/2019 6:28 pm, Stamatis Zampetakis wrote: I guess it will require some effort to setup and automate the process for supporting multiple versions but afterwards it may be easier to maintain. If the only thing that a committer has to do to update the site is committing to the master then there is not even need for a particular workflow. On Mon, Dec 9, 2019 at 10:31 PM Julian Hyde wrote: We might be inventing requirements here, in order to justify a “cool” technical change. I don’t think there is a strong requirement for multiple versions of the site. (Sure, it would be nice.) This thread started with Stamatis pointing out that it was complicated to update the site. If we support multiple versions, will this actually make things less complicated? Julian On Dec 9, 2019, at 1:23 PM, Stamatis Zampetakis wrote: In the short term we should try to do our best to follow the existing workflow. In the medium term we shall hope that things will be easier with the automated build of the website. In the longer term, I would really prefer to migrate towards a solution like the one proposed by Vladimir. As I also mentioned in a previous email, there are many projects who publish multiple versions of the doc and I find this very helpful. People usually wait some time before updating their libraries to the latest release; in this and other cases it is helpful to have a couple versions of the doc available online. On Sun, Dec 8, 2019 at 11:02 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: Francis>There are also links to Avatica docs in Francis>the side bar and it would be a bit strange to have them always point to Francis>the master version of Avatica. gradle.properties references the Avatica version, so we could print the appropriate links. Michael>that need to be made that are independent of a particular release Michael>(e.g. adding a commiter)? Michael>Would I go back and edit the previous Michael>release branch? No. You update committers on a master branch Michael>Do we somehow label parts of the site as being Michael>release-independent? It makes little sense to discuss. The answer will be obvious once someone tries. Michael>Even if this is the case, consider when we might Michael>have to correct documentation errors from a revious release The current ASF rule is to have rel/... tag for each release. That is the site build script could use rel/vX.Y tags to get "released versions". Then there are at least two strategies. a) If we want to update documentation for calcite-1.10.0, then we could release calcite-v1.10.1. b) If a "silent" update is required (e.g. fix typo), then we could invent "support/vX.Y" branches, and commit the fix to that branch. Note: the current release process does not require a "release branch". The build script does NOT create new commits to the source repository. However, we could create one on-demand (e.g. in case we really need to patch the old site version or back-port a fix) Vladimir
Re: Updating the Website
I guess it will require some effort to setup and automate the process for supporting multiple versions but afterwards it may be easier to maintain. If the only thing that a committer has to do to update the site is committing to the master then there is not even need for a particular workflow. On Mon, Dec 9, 2019 at 10:31 PM Julian Hyde wrote: > We might be inventing requirements here, in order to justify a “cool” > technical change. > > I don’t think there is a strong requirement for multiple versions of the > site. (Sure, it would be nice.) > > This thread started with Stamatis pointing out that it was complicated to > update the site. If we support multiple versions, will this actually make > things less complicated? > > Julian > > > > > On Dec 9, 2019, at 1:23 PM, Stamatis Zampetakis > wrote: > > > > In the short term we should try to do our best to follow the existing > > workflow. > > > > In the medium term we shall hope that things will be easier with the > > automated build of the website. > > > > In the longer term, I would really prefer to migrate towards a solution > > like the one proposed by Vladimir. > > As I also mentioned in a previous email, there are many projects who > > publish multiple versions of the doc and I find this very helpful. > > People usually wait some time before updating their libraries to the > latest > > release; in this and other cases it is helpful to have a couple versions > of > > the doc available online. > > > > > > On Sun, Dec 8, 2019 at 11:02 PM Vladimir Sitnikov < > > sitnikov.vladi...@gmail.com> wrote: > > > >> Francis>There are also links to Avatica docs in > >> Francis>the side bar and it would be a bit strange to have them always > >> point to > >> Francis>the master version of Avatica. > >> > >> gradle.properties references the Avatica version, so we could print the > >> appropriate links. > >> > >> Michael>that need to be made that are independent of a particular > release > >> Michael>(e.g. adding a commiter)? > >> Michael>Would I go back and edit the previous > >> Michael>release branch? > >> > >> No. You update committers on a master branch > >> > >> Michael>Do we somehow label parts of the site as being > >> Michael>release-independent? > >> > >> It makes little sense to discuss. The answer will be obvious once > someone > >> tries. > >> > >> Michael>Even if this is the case, consider when we might > >> Michael>have to correct documentation errors from a revious release > >> > >> The current ASF rule is to have rel/... tag for each release. > >> That is the site build script could use rel/vX.Y tags to get "released > >> versions". > >> > >> Then there are at least two strategies. > >> a) If we want to update documentation for calcite-1.10.0, then we could > >> release calcite-v1.10.1. > >> b) If a "silent" update is required (e.g. fix typo), then we could > invent > >> "support/vX.Y" branches, and commit the fix to that branch. > >> > >> Note: the current release process does not require a "release branch". > >> The build script does NOT create new commits to the source repository. > >> However, we could create one on-demand (e.g. in case we really need to > >> patch the old site version or back-port a fix) > >> > >> Vladimir > >> > >
Re: Updating the Website
We might be inventing requirements here, in order to justify a “cool” technical change. I don’t think there is a strong requirement for multiple versions of the site. (Sure, it would be nice.) This thread started with Stamatis pointing out that it was complicated to update the site. If we support multiple versions, will this actually make things less complicated? Julian > On Dec 9, 2019, at 1:23 PM, Stamatis Zampetakis wrote: > > In the short term we should try to do our best to follow the existing > workflow. > > In the medium term we shall hope that things will be easier with the > automated build of the website. > > In the longer term, I would really prefer to migrate towards a solution > like the one proposed by Vladimir. > As I also mentioned in a previous email, there are many projects who > publish multiple versions of the doc and I find this very helpful. > People usually wait some time before updating their libraries to the latest > release; in this and other cases it is helpful to have a couple versions of > the doc available online. > > > On Sun, Dec 8, 2019 at 11:02 PM Vladimir Sitnikov < > sitnikov.vladi...@gmail.com> wrote: > >> Francis>There are also links to Avatica docs in >> Francis>the side bar and it would be a bit strange to have them always >> point to >> Francis>the master version of Avatica. >> >> gradle.properties references the Avatica version, so we could print the >> appropriate links. >> >> Michael>that need to be made that are independent of a particular release >> Michael>(e.g. adding a commiter)? >> Michael>Would I go back and edit the previous >> Michael>release branch? >> >> No. You update committers on a master branch >> >> Michael>Do we somehow label parts of the site as being >> Michael>release-independent? >> >> It makes little sense to discuss. The answer will be obvious once someone >> tries. >> >> Michael>Even if this is the case, consider when we might >> Michael>have to correct documentation errors from a revious release >> >> The current ASF rule is to have rel/... tag for each release. >> That is the site build script could use rel/vX.Y tags to get "released >> versions". >> >> Then there are at least two strategies. >> a) If we want to update documentation for calcite-1.10.0, then we could >> release calcite-v1.10.1. >> b) If a "silent" update is required (e.g. fix typo), then we could invent >> "support/vX.Y" branches, and commit the fix to that branch. >> >> Note: the current release process does not require a "release branch". >> The build script does NOT create new commits to the source repository. >> However, we could create one on-demand (e.g. in case we really need to >> patch the old site version or back-port a fix) >> >> Vladimir >>
Re: Updating the Website
In the short term we should try to do our best to follow the existing workflow. In the medium term we shall hope that things will be easier with the automated build of the website. In the longer term, I would really prefer to migrate towards a solution like the one proposed by Vladimir. As I also mentioned in a previous email, there are many projects who publish multiple versions of the doc and I find this very helpful. People usually wait some time before updating their libraries to the latest release; in this and other cases it is helpful to have a couple versions of the doc available online. On Sun, Dec 8, 2019 at 11:02 PM Vladimir Sitnikov < sitnikov.vladi...@gmail.com> wrote: > Francis>There are also links to Avatica docs in > Francis>the side bar and it would be a bit strange to have them always > point to > Francis>the master version of Avatica. > > gradle.properties references the Avatica version, so we could print the > appropriate links. > > Michael>that need to be made that are independent of a particular release > Michael>(e.g. adding a commiter)? > Michael>Would I go back and edit the previous > Michael>release branch? > > No. You update committers on a master branch > > Michael>Do we somehow label parts of the site as being > Michael>release-independent? > > It makes little sense to discuss. The answer will be obvious once someone > tries. > > Michael>Even if this is the case, consider when we might > Michael>have to correct documentation errors from a revious release > > The current ASF rule is to have rel/... tag for each release. > That is the site build script could use rel/vX.Y tags to get "released > versions". > > Then there are at least two strategies. > a) If we want to update documentation for calcite-1.10.0, then we could > release calcite-v1.10.1. > b) If a "silent" update is required (e.g. fix typo), then we could invent > "support/vX.Y" branches, and commit the fix to that branch. > > Note: the current release process does not require a "release branch". > The build script does NOT create new commits to the source repository. > However, we could create one on-demand (e.g. in case we really need to > patch the old site version or back-port a fix) > > Vladimir >
Re: Updating the Website
Francis>There are also links to Avatica docs in Francis>the side bar and it would be a bit strange to have them always point to Francis>the master version of Avatica. gradle.properties references the Avatica version, so we could print the appropriate links. Michael>that need to be made that are independent of a particular release Michael>(e.g. adding a commiter)? Michael>Would I go back and edit the previous Michael>release branch? No. You update committers on a master branch Michael>Do we somehow label parts of the site as being Michael>release-independent? It makes little sense to discuss. The answer will be obvious once someone tries. Michael>Even if this is the case, consider when we might Michael>have to correct documentation errors from a revious release The current ASF rule is to have rel/... tag for each release. That is the site build script could use rel/vX.Y tags to get "released versions". Then there are at least two strategies. a) If we want to update documentation for calcite-1.10.0, then we could release calcite-v1.10.1. b) If a "silent" update is required (e.g. fix typo), then we could invent "support/vX.Y" branches, and commit the fix to that branch. Note: the current release process does not require a "release branch". The build script does NOT create new commits to the source repository. However, we could create one on-demand (e.g. in case we really need to patch the old site version or back-port a fix) Vladimir
Re: Updating the Website
I do like the idea of having past documentation available, but I'm not sure I see how this would work. What if there are updates to the site that need to be made that are independent of a particular release (e.g. adding a commiter)? Would I go back and edit the previous release branch? Do we somehow label parts of the site as being release-independent? Even if this is the case, consider when we might have to correct documentation errors from a revious release. There may well be a path forward to make this work, but it's not obvious to me at the moment. -- Michael Mior mm...@apache.org Le dim. 8 déc. 2019 à 15:53, Vladimir Sitnikov a écrit : > > >There are changes > >such as documentation for new features, etc that should not be published > >immediately > > Could we publish "master" documentation under /master or /nightly folders? > Anyway, it makes sense to publish multiple versions of the documentation. > > So why don't we publish master documentation as well? > > For instance: > https://docs.gradle.org/nightly/userguide/userguide.html > https://docs.gradle.org/6.0.1/userguide/userguide.html > https://docs.gradle.org/5.6.4/userguide/userguide.html > > And so on. > > I suggest we drop "site" branch, and build the site from "master + release > branches (or tags) for the past releases". > Then the site update would be as follows: > - commit to master > - done > > Vladimir
Re: Updating the Website
I think this is a great idea. Users of older versions can refer to the appropriate version of the docs. If we do want to go down this path, we need to look at the content in https://calcite.apache.org/docs/ and decide what should be versioned and what should just be on master. There are also links to Avatica docs in the side bar and it would be a bit strange to have them always point to the master version of Avatica. On 9/12/2019 7:53 am, Vladimir Sitnikov wrote: There are changes such as documentation for new features, etc that should not be published immediately Could we publish "master" documentation under /master or /nightly folders? Anyway, it makes sense to publish multiple versions of the documentation. So why don't we publish master documentation as well? For instance: https://docs.gradle.org/nightly/userguide/userguide.html https://docs.gradle.org/6.0.1/userguide/userguide.html https://docs.gradle.org/5.6.4/userguide/userguide.html And so on. I suggest we drop "site" branch, and build the site from "master + release branches (or tags) for the past releases". Then the site update would be as follows: - commit to master - done Vladimir
Re: Updating the Website
>There are changes >such as documentation for new features, etc that should not be published >immediately Could we publish "master" documentation under /master or /nightly folders? Anyway, it makes sense to publish multiple versions of the documentation. So why don't we publish master documentation as well? For instance: https://docs.gradle.org/nightly/userguide/userguide.html https://docs.gradle.org/6.0.1/userguide/userguide.html https://docs.gradle.org/5.6.4/userguide/userguide.html And so on. I suggest we drop "site" branch, and build the site from "master + release branches (or tags) for the past releases". Then the site update would be as follows: - commit to master - done Vladimir
Re: Updating the Website
I agree, updating the website is a lot more complicated than it should be. However, the site branch is necessary because it's used to publish changes that must be visible on the site immediately. There are changes such as documentation for new features, etc that should not be published immediately, so we don't cherry-pick them into the site branch. So, in summary: - Changes to the website should always be committed to master. - If the change must be visible right now, cherry pick the commit into the site branch. - Once a final release of Calcite has been tagged, reset the site branch to master: git reset --hard master From personal experience, it's the building and committing of the generated website files that is the most troublesome. Due to slightly different versions of Jekyll and timezones, the generated files might change slightly and a simple update to the website generates a cascade of changes across files that were not touched during the editing process. There is then the challenge of pulling down the calcite-site repo and merging those changes in, while being careful not to delete things like the javadocs. I am almost very close to getting the automated website builds sorted. I was in contact with infra last month to get the token to push to calcite-site added to our Github Actions secrets and will ping them again this week. Once automated website builds are implemented, editing the website should be as easy as: - Run "docker-compose run --service-ports dev" to start a docker container for live reloading. - Make your changes. - Visit http://localhost:4000 and reload the page as you make changes to see the results. - Commit your changes. - If the change needs to be published immediately, cherry-pick the commit into the site branch and push. Vladimir, as part of the release process, the site branch and the master branch should be even once a final release is tagged. This ensures that any changes to the site that were not published previously because they relate to changes in the code base would finally be published. Do you think it would be feasible to update the publishDist gradle task to even the site and master branches using git reset --hard master? Francis On 8/12/2019 8:19 pm, Vladimir Sitnikov wrote: As I was trying to update the website recently I noticed that there is a mixture of all kind of commits The same goes for me. The rebase-cherrypick dance seems to be more complicated than it should. Vladimir
Re: Updating the Website
>As I was trying to update the website recently I noticed that there is a >mixture of all kind of commits The same goes for me. The rebase-cherrypick dance seems to be more complicated than it should. Vladimir
