Christopher, In your breakdown of my proposal, I think there's a miscommunication in regard to the number of build steps and branches needed to maintain.
Looking back I should have used descriptors like Branch C, Branch S, Branch P vs main and staging due to the collision with the current branch names in use. Feature branches would feed into staging and staging would be a direct merge into the site branch for a total of three branches. One of those branches is deleted after merge, resulting in two long-lived branches. One for the staging site, and one for the production site. The number of build job runs should remain exactly the same as our current implementation. The only change would be an additional PR and the migration of the publish operation from a local action to a github deploy action. Unless link paths need to be regenerated for the new site location, there shouldn't need to be an additional jekyll build run on the merge from the staging site to the production site branch. Since the additional PR is what would be used for formal review, the amount of work generated for additional team members remains the same, with the benefit that changes are deployed directly to the production site once the review is complete. I agree with the downside of merge commits in my proposal and that fast-forward merges are better in terms of git history management. While this could be solved with changing the merge strategy of the repository, I understand the desire to not involve the infra team. However, the majority of change tracking problems that come from extra merge commits being introduced don't exist in this scenario as multiple versions of the website aren't supported. My personal opinion is that fast-forward merges do not provide enough benefit in this scenario to outway a build pipeline that eliminates maintaining a local development environment for site updates. *Current thoughts* I agree with the desire to streamline the build pipeline and move the production publish stage from a manual step to an automated build action. Having a staging area is useful in the event any large scale changes need to be made to the site like major rendering version updates. But I do agree that it provides little benefit when introducing minor changes outside of the possibility of discovering link/navigation issues. Regardless of what we do with the staging site, I don't think the answer is do nothing. I think one of the things we've identified in this discussion is the need for link validation to run in our QA build step. That seems to be the most common bug that makes it into the production site so identifying and eliminating those errors reduces the need for staging to exist. - Dan R. On Mon, Apr 17, 2023 at 11:22 AM Keith Turner <ke...@deenlo.com> wrote: > On Wed, Apr 12, 2023 at 12:34 PM Christopher <ctubb...@apache.org> wrote: > > > > I don't think Dan's suggested flow is much different than what we have > > now. In both cases, the steps are: > > > > 1. Initial PR to a branch (currently against main; Dan's approach > > would be against a separate main-like staging branch) > > 2. Auto build on PR merge (currently, that's main -> asf-staging; > > Dan's approach would build staging -> asf-staging) > > 3. View on staging site to evaluate for needed changes (auto-publish > > of asf-staging branch) and repeat steps 1-3 > > 4. Then decision to publish (currently, that's fast-forward git merge > > from asf-staging to asf-site branch; in Dan's approach, it would be a > > merge from the staging branch to the main branch, which triggers > > another Jekyll build to the asf-site branch) > > 5. View on published site to evaluate for needed changes and repeat > steps 1-5 > > > > In Dan's approach, we would be maintaining two source branches and two > > build branches, and there's an extra Jekyll build, and the possibility > > of introducing extra merge commits if we fail to do a FF merge from > > the staging branch to the main branch. > > > > Overall, I don't think the approach gets us anything we don't already > > have, and it uses a bit more resources. > > If we really need the extra staging step, then I'd prefer to keep the > > current flow. > > > > For comparison, the streamlined process I'm proposing is: > > > > 1. Initial PR to main branch > > 2. Auto build and publish on PR merge > > 3. View on published site to evaluate if additional changes need to be > > made and repeat steps 1-3 > > > > We could delete the asf-staging branch, operating only with the main > > and asf-site branches. In all cases, current and proposed, there are > > alternate ways to view/evaluate a change: > > > > 1. As rendered Markdown in the GitHub PR (navigation and Jekyll > > templated code won't work) > > 2. As local build (requires user to set up a local environment) > > 3. As built artifacts from GitHub Actions (slow download; best viewed > > in a local webserver, but can also be viewed directly in a browser, > > with some navigation limitations) > > 4. Temporarily create a new staging branch to stage to a separate > > named staging site > > 5. Just view it published at the main site and fix it in a subsequent > > change because it's fine if the site is broken slightly for a brief > > moment (we can always revert the change if a fix isn't quick) > > > > So, to recap the positions of this discussion's participants: > > > > - Chris and I are in favor of streamlining by getting rid of the staging > step. > > - Dave and Dan were interested in leaving in a staging step. > > - Dan suggested an alternate staging flow. > > > > Unless others weigh in to lean one way or another, or Dave or Dan > > switch their opinions, it's kind of a toss-up (no consensus), and I > > guess we'll just keep things as they are now. > > Currently the website README makes it really easy to know what to do > and its really quick to follow the instructions. So I do not see > staging as a burden. > > I do not recall an instance where I discovered an error in staging > before updating. Staging does give us a buffer to prevent us from > breaking the website, but I think if we ever did break it that we > could also fix it quickly. > > Not seeing staging as a burden, but not having found it useful leaves > me slightly in favor of removing staging. If we didn't have such a > good README for the website I would be more strongly in favor of > removing staging. > > > > > > > > > > On Sun, Apr 9, 2023 at 9:56 AM Dave Marion <dmario...@gmail.com> wrote: > > > > > > Christopher, > > > > > > IIRC the local build is likely sufficient. It may have been that I > failed > > > to use it in all cases. I do remember in the past merging some PR's > then > > > noticing on the staging site that the links or formatting were not > right. I > > > think Dan's suggested change to the git workflow might also be useful > > > (Auto-deply to staging on merge to staging branch, auto-deploy to > apache.org > > > on merge to main). > > > > > > On Fri, Apr 7, 2023 at 10:03 PM Christopher <ctubb...@apache.org> > wrote: > > > > > > > Hi Dave, > > > > > > > > How often would you estimate the diff in the PR is insufficient, and > > > > you need a full site build to get a sense of how the change is > viewed? > > > > I imagine this might come up with some longer blog posts with a lot > of > > > > complex layout and references to docs, etc., but that doesn't happen > > > > very frequently. I don't know if there are other times you feel it > > > > would be helpful or how often those occur. > > > > > > > > Aside from local builds, are your concerns mitigated in any way by > the > > > > following? > > > > > > > > 1. Code reviewers checking the proposed changes for layout-related > > > > mistakes before merging, or > > > > 2. Subsequent commits to correct initial publication mistakes would > > > > also be streamlined. > > > > > > > > (For reference, the local build is documented in the website README > at > > > > > > > > > https://github.com/apache/accumulo-website/blob/main/README.md#local-builds-for-testing > > > > ) > > > > > > > > On Fri, Apr 7, 2023 at 7:24 PM Dave Marion <dmario...@gmail.com> > wrote: > > > > > > > > > > I find the staging site useful for reviewing changes before > publishing. > > > > > There are some things that are not the easiest to review in a > markdown > > > > > diff. Especially since the markdown is transformed to what is > ultimately > > > > > displayed. I have no issue with streamlining this, I'll just need > to > > > > > remember to be more careful. There is a way to deploy locally > IIRC, I'll > > > > > just have to use that. > > > > > > > > > > On Fri, Apr 7, 2023 at 6:34 PM Christopher Shannon < > > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > > > +1 to make the change. I agree that when PRs are merged we > generally > > > > just > > > > > > want to publish immediately. > > > > > > > > > > > > FWIW, this is how we do things for the ActiveMQ website > > > > > > <https://github.com/apache/activemq-website> and it works fine, > we > > > > just > > > > > > publish and there is no staging. > > > > > > > > > > > > On Fri, Apr 7, 2023 at 3:49 PM Christopher <ctubb...@apache.org> > > > > wrote: > > > > > > > > > > > > > Hi Accumulo Devs, > > > > > > > > > > > > > > When I first set up the automation using .asf.yaml for Jekyll > builds > > > > > > > to go to the asf-staging branch, I expected us to get a bit > more use > > > > > > > out of the staging site at https://accumulo.staged.apache.org/ > > > > > > > However, that really hasn't happened, and it seems that we > basically > > > > > > > almost always want to publish to the > https://accumulo.apache.org > > > > once > > > > > > > we've approved a PR against the main branch and have a > successful > > > > > > > Jekyll build. > > > > > > > Having the staging site is now just an unnecessary extra step > to > > > > > > > publish (that we sometimes forget to do for a while, even > though it's > > > > > > > trivial to run ./_scripts/publish.sh). I'm not sure it's > adding any > > > > > > > value. > > > > > > > > > > > > > > Perhaps we should just have Jekyll build directly to the > asf-site > > > > > > > branch and get rid of the asf-staging branch? > > > > > > > > > > > > > > The one benefit to the staging site is that, in theory, it's a > > > > > > > conscientious decision to publish. But, I don't think we've > ever come > > > > > > > across any kind of situation where we wanted to stage the > site, but > > > > > > > then didn't immediately want to publish it. So, I'm not > convinced > > > > that > > > > > > > extra step is adding any value, especially since we're almost > always > > > > > > > making the conscientious decision in GitHub when approving a > PR to > > > > > > > update the site already. > > > > > > > > > > > > > > Any thoughts or opinions on this? I'm leaning slightly towards > > > > > > > streamlining this. > > > > > > > > > > > > > > Regards, > > > > > > > Christopher > > > > > > > > > > > > > > > > > >
