On 30 April 2018 at 19:24, Hervé BOUTEMY <herve.bout...@free.fr> wrote: > Le lundi 30 avril 2018, 12:46:09 CEST sebb a écrit : >> On 30 April 2018 at 06:56, Hervé BOUTEMY <herve.bout...@free.fr> wrote: >> > Le lundi 30 avril 2018, 03:19:20 CEST sebb a écrit : >> >> On 29 April 2018 at 22:11, Hervé BOUTEMY <herve.bout...@free.fr> wrote: >> >> > better than a discussion: a demo >> >> > >> >> > here is the website: >> >> > https://asf-attic.github.io/ >> >> > >> >> > its source: >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/source >> >> > its output: >> >> > https://github.com/asf-attic/asf-attic.github.io/tree/master >> >> > >> >> > the only thing that is not included here is the code for the CI to >> >> > checkout >> >> > the output branch and update content after rebuild >> >> >> >> All I see is two disjoint branches. >> > >> > yes, that's what it's about >> > >> >> It's not obvious how a user is supposed to update the master from the >> >> source. >> > >> > people who know GitHub pages know: that's now part of common culture for >> > many people >> >> So how do people update the site? >> >> Is it enough to edit the source branch and commit the change? > yes, once a CI job is configured (like you do with Buildbot, and I do usually > with Jenkins + Maven scm-publish) > >> >> > and from an operational point of view, that's why in general there is a CI >> > server doing the official build from source branch then committing output >> > to output branch >> >> Is that part of Github's offering? > if you use the GitHub Jekyll, yes, the build is magic > if you want your own build engine, it's up to you to do the CI setup > >> >> >> The build.sh script can be used to create a docs/ tree, but then what? >> > >> > when you build locally, it's just for yourself, because you want to check >> > a >> > change that is not as trivial as usual >> > then the change in output branch is usually done by a CI server >> > >> > When used with Maven, there is scm-pulbish plugin for that: it's a common >> > convention, and that's what is used by Apache Cayenne for example. >> > >> > When used with Jekyll or anything else as rendering engine, I don't know >> > how people script the output commit: Apache Freemarker just tells in >> > their documentation "To publish the built site, commit the output into >> > the "asf- site" branch". And Apache Accumulo writes a serie of shell >> > commands that they tell are launched by a Git hook (and not a CI server). >> >> AFAICT that only commits the change to the asf-site branch; it does >> not push the updated branch. > perhaps instructions are not fully written: in general, a CI job does the work > >> >> > And again, in general, there is a central official setup does the job in a >> > central and official way, to avoid subtle differences when rendering from >> > multiple personal configurations >> > >> >> > But what is done here with GitHub GitPubSub equivalent can be done >> >> > exactly >> >> > the same way at Apache Software Foundation >> >> I'm not disputing that, but if I am to amend the Buildbot to mimic the >> GitHub behaviour I need to know what it does. > sure :) > it's about doing a git commit + push to the output branch, like it did > previously the svn commit to svnpubsub location > > the demo on GitHub is just here to see it in a very concrete way, since GitHub > provides the gitwcsub to publish from the branch to the web server
Ah, OK. I don't seem to be able to edit the source, so I cannot try it out. But if I did, I assume I would see my changes applied to the source and then master branches, is that correct? >> >> >> > Regards, >> >> > >> >> > Hervé >> >> > >> >> > Le dimanche 29 avril 2018, 19:13:43 CEST Hervé BOUTEMY a écrit : >> >> >> Le dimanche 29 avril 2018, 14:46:09 CEST sebb a écrit : >> >> >> > On 29 April 2018 at 11:33, Hervé BOUTEMY <herve.bout...@free.fr> > wrote: >> >> >> > > Le dimanche 29 avril 2018, 11:04:44 CEST sebb a écrit : >> >> >> > >> On 29 April 2018 at 09:41, Hervé BOUTEMY <herve.bout...@free.fr> >> > >> > wrote: >> >> >> > >> > first, I want to reassure everybody: this is a discussion, to >> >> >> > >> > get >> >> >> > >> > common >> >> >> > >> > knowledge of how things work in other projects then may work in >> >> >> > >> > the >> >> >> > >> > future for Attic if we decide to do an equivalent setup >> >> >> > >> >> >> >> > >> +1 >> >> >> > >> >> >> >> > >> > Le dimanche 29 avril 2018, 07:50:21 CEST sebb a écrit : >> >> >> > >> >> On 28 April 2018 at 12:48, sebb <seb...@gmail.com> wrote: >> >> >> > >> >> > On 28 April 2018 at 12:37, Hervé BOUTEMY >> >> >> > >> >> > <herve.bout...@free.fr> >> >> >> >> >> >> wrote: >> >> >> > >> >> ... >> >> >> > >> >> >> >> >> > >> >> >> In Git, this would naturally be in a separate branch named >> >> >> > >> >> >> "asf-site" >> >> >> > >> >> >> >> >> > >> >> How would that work for Attic? >> >> >> > >> >> >> >> >> > >> >> Where would the source files used to generate the site be >> >> >> > >> >> held? >> >> >> > >> > >> >> >> > >> > There are multiple ways of doing, and GitHub documented it as >> >> >> > >> > clearly >> >> >> > >> > as >> >> >> > >> > possible [2] (yes, what we do at ASF with GitPubSub is exactly >> >> >> > >> > what >> >> >> > >> > GitHub calls "GitHub Pages", with marketing bells turned on and >> >> >> > >> > technical >> >> >> > >> > details on the build solution turned off) >> >> >> > >> > >> >> >> > >> > The 2 common ways are: >> >> >> > >> > 1. publish html from separate branch (which would be by default >> >> >> > >> > "asf-site" >> >> >> > >> > at ASF, and is "gh-pages" at GitHub) 2. publish html from a >> >> >> > >> > subdirectory >> >> >> > >> > on master branch (you see Attic current pattern?) >> >> >> > >> > >> >> >> > >> > I find the first option a lot more clear from a build+scm >> >> >> > >> > perspective >> >> >> > >> > than >> >> >> > >> > the second one. This will avoid the exact same discussion we >> >> >> > >> > have >> >> >> > >> > currently at Attic with svn to know who commits the generated >> >> >> > >> > content >> >> >> > >> > (& >> >> >> > >> > when as a consequence): - CI after source-only commit? >> >> >> > >> > - or user who builds on his machine then commits simultaneously >> >> >> > >> > source >> >> >> > >> > and >> >> >> > >> > generated content? >> >> >> > >> > >> >> >> > >> > Then looking at ASF gitwcsub configuration [2], I had a look at >> >> >> > >> > many >> >> >> > >> > ASF >> >> >> > >> > projects: the 2 ways are used. I picked Cayenne [3] case to >> >> >> > >> > show a >> >> >> > >> > case >> >> >> > >> > where: >> >> >> > >> > - master branch is a source branch, with markup and a build >> >> >> > >> > script >> >> >> > >> > - asf-site branch is a completely separate branch that contains >> >> >> > >> > generated >> >> >> > >> > html It uses Maven scm-publish plugin to update asf-git branch >> >> >> > >> > with >> >> >> > >> > generated html [6] >> >> >> > >> > >> >> >> > >> > But there is also Freemarker [4], that has a simple README >> >> >> > >> > telling >> >> >> > >> > "To >> >> >> > >> > publish the built site, commit the output into the "asf-site" >> >> >> > >> > branch". >> >> >> > >> > >> >> >> > >> > Or Accumulo [5] which uses Jekyll and has some instructions to >> >> >> > >> > publish >> >> >> > >> > generated output to asf-site branch with a git-hook that I >> >> >> > >> > don't >> >> >> > >> > fully >> >> >> > >> > understand, but that maybe Attic members will prefer since it >> >> >> > >> > seems >> >> >> > >> > it's >> >> >> > >> > more the common culture here >> >> >> > >> > >> >> >> > >> > Notice: I'm a Maven guy, I co-wrote the Maven scm-publish >> >> >> > >> > plugin >> >> >> > >> > used >> >> >> > >> > by >> >> >> > >> > Cayenne, and I use it in many projects, initially with svn as >> >> >> > >> > target >> >> >> > >> > source control (in 2012, for svnpubsub & Apache CMS) then with >> >> >> > >> > git >> >> >> > >> > also, >> >> >> > >> > when GitHub pages became popular. But I see that it's not the >> >> >> > >> > right >> >> >> > >> > choice at Attic because it's not the most common Attic culture. >> >> >> > >> >> >> >> > >> Attic does not produce source code. >> >> >> > >> The only output from its SCM is the website. >> >> >> > > >> >> >> > > yes, like any other website that I showed: source code here is a >> >> >> > > markup >> >> >> > > language (be it Markdown, xdoc, static content, or anything else) >> >> >> > > Attic is really exactly the same >> >> >> > > >> >> >> > > What makes Attic different is that Attic does not have any other >> >> >> > > repo >> >> >> > > for >> >> >> > > "programming" code: that's true, but does not change anything >> >> >> > > regarding >> >> >> > > site> >> >> >> > > >> >> >> > >> AFAICT, all the above examples have a branch which contains the >> >> >> > >> source >> >> >> > >> for building the website. >> >> >> > > >> >> >> > > yes, I explained I chose them exactly for that reason >> >> >> > > >> >> >> > > I can show you Airavata site, which is quite simple and did the >> >> >> > > other >> >> >> > > choice: https://github.com/apache/airavata-site >> >> >> > > >> >> >> > > source is in source, output is in content in the same branch >> >> >> > >> >> >> > This is equivalent to Attic. >> >> >> > >> >> >> > > it could have been: source in master branch, content in asf-site >> >> >> > > branch >> >> >> > > which is the most common setup in GitHub pages (= what people >> >> >> > > nowadays >> >> >> > > know a lot) >> >> >> > > >> >> >> > >> 1) The source is edited. >> >> >> > >> 2) Run the build script to create the output in a clean >> >> >> > >> subdirectory >> >> >> > >> 3) Copy the subdirectory tree to the asf-site branch >> >> >> > >> 4) commit the asf-site branch >> >> >> > >> 5) The entire asf-site branch is then published via pubsub. >> >> >> > >> >> >> >> > >> What Attic does currently is: >> >> >> > >> 1) & 2) as above >> >> >> > >> 3) commit the changes >> >> >> > >> 5) as above >> >> >> > >> >> >> >> > >> i.e. there is no need to copy the generated output anywhere >> >> >> > >> because >> >> >> > >> it >> >> >> > >> is part of the same repo. >> >> >> > >> >> >> >> > >> This works because svnpubsub is set up to get its source from the >> >> >> > >> docs/ subdirectory >> >> >> > > >> >> >> > > yes, the setup with source and output in the same svn repo or Git >> >> >> > > branch >> >> >> > > makes it simple to checkout, but it mixes 2 types of files (source >> >> >> > > and >> >> >> > > generated) >> >> >> > > >> >> >> > > separating source and generated in 2 separate locations (separate >> >> >> > > svn >> >> >> > > root >> >> >> > > or different branches in the same git repo) makes things more >> >> >> > > clear, >> >> >> > > at >> >> >> > > the cost of an extra step to check out the generated content then >> >> >> > > update >> >> >> > > with the updated content >> >> >> > >> >> >> > The workspace still contains both source and generated output in the >> >> >> > examples I have seen. >> >> >> > >> >> >> > I assume it is ignored by SVN/Git so does not get committed or show >> >> >> > up >> >> >> > as a local change. >> >> >> >> >> >> I showed you Cayenne, Freemarker and Accumulo that are not like this. >> >> >> Here we go back to Freemarker: >> >> >> - source: https://github.com/apache/freemarker-site >> >> >> - output: https://github.com/apache/freemarker-site/tree/asf-site >> >> >> >> >> >> > > yes: choose your issue >> >> >> > > personally, I prefer the second setup (clear but a little harder >> >> >> > > to >> >> >> > > setup) >> >> >> > > I don't like having mixed content in one repo (source and >> >> >> > > generated >> >> >> > > output) >> >> >> > > >> >> >> > > If everybody understands that these 2 setups a completely >> >> >> > > equivalent >> >> >> > > but >> >> >> > > really prefer the mixed one (just to avoid a second checkout), >> >> >> > > I'll >> >> >> > > let >> >> >> > > you >> >> >> > > go: I don't have any problem myself, I make a strong difference >> >> >> > > between >> >> >> > > source directory and output directory >> >> >> > >> >> >> > There's still mixed content in local workspaces unless you generate >> >> >> > the output in a separate tree. >> >> >> > >> >> >> > > But if people start to edit output directory instead of source >> >> >> > > (like >> >> >> > > it >> >> >> > > is >> >> >> > > so easy to do in the mixed content setup), you're at risk >> >> >> > >> >> >> > It's also possible to checkin the generated output if it's not >> >> >> > properly ignored in a 2 branch version. >> >> >> > And then wonder why the site does not get updated. >> >> >> >> >> >> that's why in general there is a .gitignore or svn:ignore that is >> >> >> properly >> >> >> configured >> >> >> >> >> >> > >> I don't know if gitpubsub can take its input from a subdirectory >> >> >> > >> of >> >> >> > >> a >> >> >> > >> branch. >> >> >> > > >> >> >> > > it can: see Airavata site >> >> >> > >> >> >> > Ah - I see now. >> >> >> > >> >> >> > The webserver defines the site to be under content/, so the branch >> >> >> > can >> >> >> > contain other files in parallel directories. >> >> >> > >> >> >> > >> If not, then we will have to change strategy in order to use Git. >> >> >> > >> Otherwise, we have a choice. >> >> >> > > >> >> >> > > we have a choice >> >> >> > >> >> >> > Yes. >> >> >> > >> >> >> > I prefer the status quo, not least because it involves fewer changes >> >> >> > (I think only renaming docs/ to content/ if we move to Git). >> >> >> > Using multiple repos would involve updating instructions as well. >> >> >> >> >> >> no, it's not multiple repos but multiple branches of the same repo >> >> >> >> >> >> > But if the majority want to change I won't object. >> >> >> >> >> >> since you are the guy who does the buidbot script, that does the >> >> >> commit, >> >> >> you'll have to be confident that you can code the multi-branch option >> >> >> >> >> >> > > Regards, >> >> >> > > >> >> >> > > Hervé >> >> >> > > >> >> >> > >> > Regards, >> >> >> > >> > >> >> >> > >> > Hervé >> >> >> > >> > >> >> >> > >> > >> >> >> > >> > [1] >> >> >> > >> > https://help.github.com/articles/configuring-a-publishing-sourc >> >> >> > >> > e-f >> >> >> > >> > or-> > >> > gi th >> >> >> > >> > ub-pages/ >> >> >> > >> > >> >> >> > >> > [2] >> >> >> > >> > https://github.com/apache/infrastructure-puppet/blob/deployment >> >> >> > >> > /mo >> >> >> > >> > dul >> >> >> > >> > es >> >> >> > >> > /g >> >> >> > >> > itwcsub/files/config/gitwcsub.cfg >> >> >> > >> > >> >> >> > >> > [3] https://github.com/apache/cayenne-website/ >> >> >> > >> > >> >> >> > >> > [4] https://github.com/apache/freemarker-site >> >> >> > >> > >> >> >> > >> > [5] https://github.com/apache/accumulo-website >> >> >> > >> > >> >> >> > >> > [6] >> >> >> > >> > https://maven.apache.org/plugins/maven-scm-publish-plugin/vario >> >> >> > >> > us-> >> > >> > tip s. >> >> >> > >> > ht >> >> >> > >> > ml#Git_branch > >
