Yes, I think the Docker image for local staging is sufficient. I'm good with removing the staging site if the Docker PR is merged.
On Mon, Apr 24, 2023 at 11:18 PM Christopher <ctubb...@apache.org> wrote: > We're still polishing up the README and other changes in the latest > version of that PR, through some iterations of code review, but I > think we'll be ready to merge it soon. That will provide one more > option for people to do jekyll development locally for the website. > > Dave also had some reservations about removing the staging site (his > last comment was a bit ambiguous "IIRC the local build is likely > sufficient"). Before I do anything to streamline website publication > by removing the mandatory staging site, I would like consensus from > everybody who had reservations. So, Dave, what do you think about > having the Docker build as an option for local staging? Would that be > enough for you to get on board with removing the mandatory staging > site step? > > On Thu, Apr 20, 2023 at 12:01 AM Christopher <ctubb...@apache.org> wrote: > > > > I reviewed your PR. It includes changes that are likely to break in my > > dev environment (the gem updates, due to errors building the native > > extensions for transitive dependencies in my environment) and make it > > harder for me to finish the 1.10.3 release changes, instead of make it > > easier... which is what my original goal was in streamlining this. I'm > > okay with having a Dockerfile... but the other changes are probably > > going to cause more problems for us in the short term than they solve. > > > > On Mon, Apr 17, 2023 at 6:11 PM Daniel Roberts <ddani...@gmail.com> > wrote: > > > > > > Christopher, > > > > > > I added a containerized dev environment in #384 which uses Jekyll's > polling > > > option to dynamically pick up changes. > > > > > > Based on that change being included, I'm ok with removing the staging > site > > > as PRs can be reviewed locally with minimal concern for dev environment > > > drift. > > > > > > Thanks, > > > > > > - Dan > > > > > > On Mon, Apr 17, 2023, 4:07 PM Christopher <ctubb...@apache.org> wrote: > > > > > > > Keith - If we made the change, the README would still exist, but it > > > > would get a bit smaller, as there would be fewer mandatory things to > > > > do. > > > > > > > > Dan - Streamlining would just remove the staging site as a mandatory > > > > step. It will still remain an optional step, following steps similar > > > > to what you describe for feature branches. It's already available as > > > > an optional step today for feature branches, but we haven't had a > need > > > > to use it that way. For a long time, staging sites weren't even an > > > > option, and we got along just fine without them. Like Keith, I can't > > > > think of a single time a staging site played any kind of influential > > > > role. But still, the feature would still be available after > > > > streamlining... just optional instead of mandatory. > > > > > > > > Regarding link validation, I agree that *might* be a useful QA build > > > > step (depending on how much effort it is, return on value it > provides, > > > > how it's implemented, false positives due to network conditions and > > > > outages, etc.). But, I think that's a separate issue, and we can look > > > > into options to run some checks in the GitHub Actions, or as a > > > > periodic Jenkins job that crawls our website looking for broken > links, > > > > independently from the streamlining proposal. > > > > > > > > On Mon, Apr 17, 2023 at 1:33 PM Daniel Roberts <ddani...@gmail.com> > wrote: > > > > > > > > > > Christopher, > > > > > > > > > > In your breakdown of my proposal, I think there's a > miscommunication in > > > > > regard to the number of build steps and branches needed to > maintain. > > > > > > > > > > Looking back I should have used descriptors like Branch C, Branch > S, > > > > Branch > > > > > P vs main and staging due to the collision with the current branch > names > > > > in > > > > > use. > > > > > > > > > > Feature branches would feed into staging and staging would be a > direct > > > > > merge into the site branch for a total of three branches. One of > those > > > > > branches is deleted after merge, resulting in two long-lived > branches. > > > > One > > > > > for the staging site, and one for the production site. > > > > > > > > > > The number of build job runs should remain exactly the same as our > > > > current > > > > > implementation. > > > > > The only change would be an additional PR and the migration of the > > > > publish > > > > > operation from a local action to a github deploy action. > > > > > > > > > > Unless link paths need to be regenerated for the new site > location, there > > > > > shouldn't need to be an additional jekyll build run on the merge > from the > > > > > staging site to the production site branch. > > > > > > > > > > Since the additional PR is what would be used for formal review, > the > > > > amount > > > > > of work generated for additional team members remains the same, > with the > > > > > benefit that changes are deployed directly to the production site > once > > > > the > > > > > review is complete. > > > > > > > > > > I agree with the downside of merge commits in my proposal and > > > > > that fast-forward merges are better in terms of git history > management. > > > > > > > > > > While this could be solved with changing the merge strategy of the > > > > > repository, I understand the desire to not involve the infra team. > > > > > > > > > > However, the majority of change tracking problems that come from > extra > > > > > merge commits being introduced don't exist in this scenario as > multiple > > > > > versions of the website aren't supported. > > > > > > > > > > My personal opinion is that fast-forward merges do not provide > enough > > > > > benefit in this scenario to outway a build pipeline that eliminates > > > > > maintaining a local development environment for site updates. > > > > > > > > > > > > > > > *Current thoughts* > > > > > I agree with the desire to streamline the build pipeline and move > the > > > > > production publish stage from a manual step to an automated build > action. > > > > > > > > > > Having a staging area is useful in the event any large scale > changes need > > > > > to be made to the site like major rendering version updates. But I > do > > > > agree > > > > > that it provides little benefit when introducing minor changes > outside of > > > > > the possibility of discovering link/navigation issues. > > > > > > > > > > Regardless of what we do with the staging site, I don't think the > answer > > > > is > > > > > do nothing. > > > > > I think one of the things we've identified in this discussion is > the need > > > > > for link validation to run in our QA build step. > > > > > > > > > > That seems to be the most common bug that makes it into the > production > > > > site > > > > > so identifying and eliminating those errors reduces the need for > staging > > > > to > > > > > exist. > > > > > > > > > > - Dan R. > > > > > > > > > > On Mon, Apr 17, 2023 at 11:22 AM Keith Turner <ke...@deenlo.com> > wrote: > > > > > > > > > > > On Wed, Apr 12, 2023 at 12:34 PM Christopher < > ctubb...@apache.org> > > > > wrote: > > > > > > > > > > > > > > I don't think Dan's suggested flow is much different than what > we > > > > have > > > > > > > now. In both cases, the steps are: > > > > > > > > > > > > > > 1. Initial PR to a branch (currently against main; Dan's > approach > > > > > > > would be against a separate main-like staging branch) > > > > > > > 2. Auto build on PR merge (currently, that's main -> > asf-staging; > > > > > > > Dan's approach would build staging -> asf-staging) > > > > > > > 3. View on staging site to evaluate for needed changes > (auto-publish > > > > > > > of asf-staging branch) and repeat steps 1-3 > > > > > > > 4. Then decision to publish (currently, that's fast-forward > git merge > > > > > > > from asf-staging to asf-site branch; in Dan's approach, it > would be a > > > > > > > merge from the staging branch to the main branch, which > triggers > > > > > > > another Jekyll build to the asf-site branch) > > > > > > > 5. View on published site to evaluate for needed changes and > repeat > > > > > > steps 1-5 > > > > > > > > > > > > > > In Dan's approach, we would be maintaining two source branches > and > > > > two > > > > > > > build branches, and there's an extra Jekyll build, and the > > > > possibility > > > > > > > of introducing extra merge commits if we fail to do a FF merge > from > > > > > > > the staging branch to the main branch. > > > > > > > > > > > > > > Overall, I don't think the approach gets us anything we don't > already > > > > > > > have, and it uses a bit more resources. > > > > > > > If we really need the extra staging step, then I'd prefer to > keep the > > > > > > > current flow. > > > > > > > > > > > > > > For comparison, the streamlined process I'm proposing is: > > > > > > > > > > > > > > 1. Initial PR to main branch > > > > > > > 2. Auto build and publish on PR merge > > > > > > > 3. View on published site to evaluate if additional changes > need to > > > > be > > > > > > > made and repeat steps 1-3 > > > > > > > > > > > > > > We could delete the asf-staging branch, operating only with > the main > > > > > > > and asf-site branches. In all cases, current and proposed, > there are > > > > > > > alternate ways to view/evaluate a change: > > > > > > > > > > > > > > 1. As rendered Markdown in the GitHub PR (navigation and Jekyll > > > > > > > templated code won't work) > > > > > > > 2. As local build (requires user to set up a local environment) > > > > > > > 3. As built artifacts from GitHub Actions (slow download; best > viewed > > > > > > > in a local webserver, but can also be viewed directly in a > browser, > > > > > > > with some navigation limitations) > > > > > > > 4. Temporarily create a new staging branch to stage to a > separate > > > > > > > named staging site > > > > > > > 5. Just view it published at the main site and fix it in a > subsequent > > > > > > > change because it's fine if the site is broken slightly for a > brief > > > > > > > moment (we can always revert the change if a fix isn't quick) > > > > > > > > > > > > > > So, to recap the positions of this discussion's participants: > > > > > > > > > > > > > > - Chris and I are in favor of streamlining by getting rid of > the > > > > staging > > > > > > step. > > > > > > > - Dave and Dan were interested in leaving in a staging step. > > > > > > > - Dan suggested an alternate staging flow. > > > > > > > > > > > > > > Unless others weigh in to lean one way or another, or Dave or > Dan > > > > > > > switch their opinions, it's kind of a toss-up (no consensus), > and I > > > > > > > guess we'll just keep things as they are now. > > > > > > > > > > > > Currently the website README makes it really easy to know what > to do > > > > > > and its really quick to follow the instructions. So I do not see > > > > > > staging as a burden. > > > > > > > > > > > > I do not recall an instance where I discovered an error in > staging > > > > > > before updating. Staging does give us a buffer to prevent us from > > > > > > breaking the website, but I think if we ever did break it that we > > > > > > could also fix it quickly. > > > > > > > > > > > > Not seeing staging as a burden, but not having found it useful > leaves > > > > > > me slightly in favor of removing staging. If we didn't have > such a > > > > > > good README for the website I would be more strongly in favor of > > > > > > removing staging. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Sun, Apr 9, 2023 at 9:56 AM Dave Marion < > dmario...@gmail.com> > > > > wrote: > > > > > > > > > > > > > > > > Christopher, > > > > > > > > > > > > > > > > IIRC the local build is likely sufficient. It may have > been that > > > > I > > > > > > failed > > > > > > > > to use it in all cases. I do remember in the past merging > some PR's > > > > > > then > > > > > > > > noticing on the staging site that the links or formatting > were not > > > > > > right. I > > > > > > > > think Dan's suggested change to the git workflow might also > be > > > > useful > > > > > > > > (Auto-deply to staging on merge to staging branch, > auto-deploy to > > > > > > apache.org > > > > > > > > on merge to main). > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 10:03 PM Christopher < > ctubb...@apache.org> > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi Dave, > > > > > > > > > > > > > > > > > > How often would you estimate the diff in the PR is > insufficient, > > > > and > > > > > > > > > you need a full site build to get a sense of how the > change is > > > > > > viewed? > > > > > > > > > I imagine this might come up with some longer blog posts > with a > > > > lot > > > > > > of > > > > > > > > > complex layout and references to docs, etc., but that > doesn't > > > > happen > > > > > > > > > very frequently. I don't know if there are other times you > feel > > > > it > > > > > > > > > would be helpful or how often those occur. > > > > > > > > > > > > > > > > > > Aside from local builds, are your concerns mitigated in > any way > > > > by > > > > > > the > > > > > > > > > following? > > > > > > > > > > > > > > > > > > 1. Code reviewers checking the proposed changes for > > > > layout-related > > > > > > > > > mistakes before merging, or > > > > > > > > > 2. Subsequent commits to correct initial publication > mistakes > > > > would > > > > > > > > > also be streamlined. > > > > > > > > > > > > > > > > > > (For reference, the local build is documented in the > website > > > > README > > > > > > at > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/accumulo-website/blob/main/README.md#local-builds-for-testing > > > > > > > > > ) > > > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 7:24 PM Dave Marion < > dmario...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > I find the staging site useful for reviewing changes > before > > > > > > publishing. > > > > > > > > > > There are some things that are not the easiest to review > in a > > > > > > markdown > > > > > > > > > > diff. Especially since the markdown is transformed to > what is > > > > > > ultimately > > > > > > > > > > displayed. I have no issue with streamlining this, I'll > just > > > > need > > > > > > to > > > > > > > > > > remember to be more careful. There is a way to deploy > locally > > > > > > IIRC, I'll > > > > > > > > > > just have to use that. > > > > > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 6:34 PM Christopher Shannon < > > > > > > > > > > christopher.l.shan...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > +1 to make the change. I agree that when PRs are > merged we > > > > > > generally > > > > > > > > > just > > > > > > > > > > > want to publish immediately. > > > > > > > > > > > > > > > > > > > > > > FWIW, this is how we do things for the ActiveMQ website > > > > > > > > > > > <https://github.com/apache/activemq-website> and it > works > > > > fine, > > > > > > we > > > > > > > > > just > > > > > > > > > > > publish and there is no staging. > > > > > > > > > > > > > > > > > > > > > > On Fri, Apr 7, 2023 at 3:49 PM Christopher < > > > > ctubb...@apache.org> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > Hi Accumulo Devs, > > > > > > > > > > > > > > > > > > > > > > > > When I first set up the automation using .asf.yaml > for > > > > Jekyll > > > > > > builds > > > > > > > > > > > > to go to the asf-staging branch, I expected us to > get a bit > > > > > > more use > > > > > > > > > > > > out of the staging site at > > > > https://accumulo.staged.apache.org/ > > > > > > > > > > > > However, that really hasn't happened, and it seems > that we > > > > > > basically > > > > > > > > > > > > almost always want to publish to the > > > > > > https://accumulo.apache.org > > > > > > > > > once > > > > > > > > > > > > we've approved a PR against the main branch and have > a > > > > > > successful > > > > > > > > > > > > Jekyll build. > > > > > > > > > > > > Having the staging site is now just an unnecessary > extra > > > > step > > > > > > to > > > > > > > > > > > > publish (that we sometimes forget to do for a while, > even > > > > > > though it's > > > > > > > > > > > > trivial to run ./_scripts/publish.sh). I'm not sure > it's > > > > > > adding any > > > > > > > > > > > > value. > > > > > > > > > > > > > > > > > > > > > > > > Perhaps we should just have Jekyll build directly to > the > > > > > > asf-site > > > > > > > > > > > > branch and get rid of the asf-staging branch? > > > > > > > > > > > > > > > > > > > > > > > > The one benefit to the staging site is that, in > theory, > > > > it's a > > > > > > > > > > > > conscientious decision to publish. But, I don't > think we've > > > > > > ever come > > > > > > > > > > > > across any kind of situation where we wanted to > stage the > > > > > > site, but > > > > > > > > > > > > then didn't immediately want to publish it. So, I'm > not > > > > > > convinced > > > > > > > > > that > > > > > > > > > > > > extra step is adding any value, especially since > we're > > > > almost > > > > > > always > > > > > > > > > > > > making the conscientious decision in GitHub when > approving > > > > a > > > > > > PR to > > > > > > > > > > > > update the site already. > > > > > > > > > > > > > > > > > > > > > > > > Any thoughts or opinions on this? I'm leaning > slightly > > > > towards > > > > > > > > > > > > streamlining this. > > > > > > > > > > > > > > > > > > > > > > > > Regards, > > > > > > > > > > > > Christopher > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
