+1 I think this format of docs is a massive improvement on the previous, really great work! Splitting it is also a good idea.
I'm not a massive fan of putting the docs and website on different branches, I think this can obscure things. For the short term this might be a reasonable way to proceed but I think longer term if this split is to be kept, we should have separate repos or some other mechanism. Many thanks Duncan On 16 October 2017 at 10:25, Richard Downer <rich...@apache.org> wrote: > All, > > As a co-conspirator on this I am obviously +1 to it :-) But it would be > good to hear some more POVs on this. > > Thomas and I (mostly Thomas) have come up with a solution which we believe > has "feature parity" with the current docs solution, except for some cases > where we believe the feature is not significant (or actually an > anti-feature). This solution provides something which is easier to > navigate, easier to turn into PDF, and it does all these using > off-the-shelf tools and no bespoke code (no more Ruby-based Jekyll > plugins!) > > The new solution does look different, in that it is not a slave to our > current website design, but its styling does incorporate the key design > elements (typography, colour scheme). It is not tightly integrated with the > website as the current docs is, but we do not consider this to be a major > issue. In my personal opinion, having such tight integration turned into an > anti-feature, as it caused conflicts between the version-related and > non-version-related pages. (For an example, browse for earlier versions of > the Brooklyn documentation - suddenly there are changes to the website > styling and the illusion of close integration is lost). > > I am in favour of proceeding with this change and seeing how it goes - if > it does not work out, then we can revert back. Any documentation changes in > the affected period should be easily backportable to the old-style website. > > In addition to Thomas's branch which forks the existing brooklyn-docs and > deletes the website, I have an alternative fork which deletes the docs, > where work on the website-only tools and content can be done. With the docs > removed there's a lot of scope to remove lots of code from the tooling. In > the future we would maybe want to look at further changes to the website, > such as upgrading to a newer version of Jekyll and investigating newer > Jekyll plugins, with the aim of removing all our bespoke code. > > Richard. > > > On 15 October 2017 at 17:20, Thomas Bouron <thomas.bouron@cloudsoftcorp. > com> > wrote: > > > Hi all. > > > > I just pushed the last set of commits to my fork for the docs[1]. This > now > > contains the docs generated by gitbook and a build script[2] to generate > > the javadoc at the right place (i.e. misc/javadoc). The last commit[3] > > updates the README to include the updated release instructions. With > this, > > I think we are now ready to go. > > > > In terms of next steps, Richard suggested to have the codebase for > website > > and docs in 2 separate branches. This is how GitHub pages works for > example > > and it makes sense to me. Unless someone is against this, I will create 2 > > branches on the git repo, `website` and `docs` and open a PR again `docs` > > on Monday. > > > > Best. > > > > [1] https://github.com/tbouron/brooklyn-docs/tree/experiment/gitbook > > [2] > > https://github.com/tbouron/brooklyn-docs/blob/experiment/ > > gitbook/javadoc/build.sh > > [3] > > https://github.com/tbouron/brooklyn-docs/commit/ > > e931560888ce51625e56e6202ead757f9d876090 > > > > On Fri, 13 Oct 2017 at 13:02 Thomas Bouron <thomas.bouron@cloudsoftcorp. > > com> > > wrote: > > > > > Hi Alex, all. > > > > > > For the past few days, I worked with Richard on this doc spike. Here > is a > > > short summary of what we have done: > > > - add collapse/expandable sections on the left menu > > > - add documentation versioning (version dropdown, top left) > > > - improve PDF output > > > - fix all internal links > > > - use brooklyn styling > > > > > > One thing left is to use YAML examples from the Brooklyn codebase. > There > > > is a plugin to do exactly that (which supports to import only a snippet > > > from a file) > > > Unfortunately, most of the doc examples exist only in the docs repo > > > therefore I think we should resolve this later on, as an incremental > > change. > > > > > > I published the new version on the same URL as before[1], hope you will > > > like it. Spoiler alert: I do :) > > > > > > PS: I created a gitbook plugin[2] to auto-generate menus on pages like > > > this one[3]. However, I had a chat with a maintainer (founder?) of > > Gitbook > > > who told me[4] that version 4 will include, out of the box, most > plugins > > we > > > currently use! They have a preview[5] of what is does. > > > > > > Best. > > > > > > [1] https://tbouron.github.io/brooklyn-docs > > > [2] https://github.com/tbouron/gitbook-plugin-partial-summary > > > [3] https://tbouron.github.io/brooklyn-docs/start/ > > > [4] > > > https://gitbook-community.slack.com/archives/C0B5XG0GK/ > p1507893620000134 > > > [5] https://betadocs.gitbook.com/features > > > > > > On Tue, 10 Oct 2017 at 12:20 Alex Heneveld < > > > alex.henev...@cloudsoftcorp.com> wrote: > > > > > >> > > >> Thomas- > > >> > > >> Had a deeper look -- gitbook has moved things forward a lot. Sounds > like > > >> it will let us throw away a lot of our home-grown docs-building and > > >> toc-building code and have good search. Look forward to seeing how it > > >> shapes up with styling and guide-v-website integration. > > >> > > >> Best > > >> Alex > > >> > > >> > > >> On 09/10/2017 09:54, Thomas Bouron wrote: > > >> > Thanks Mark. > > >> > > > >> > Regarding maintenance, it will be as easy as the current version. > > >> Updating > > >> > docs means updating markdown files. Adding/moving pages requires to > > >> modify > > >> > the `SUMMARY.md` but that's it. > > >> > One really cool thing is that Gitbook is a node app: really simple > to > > >> > install/run compare to our current solution which runs only on an > old > > >> > version of ruby => no more pain of using different versions of ruby > on > > >> your > > >> > environment. > > >> > > > >> > In terms of feature gaps, Gitbook provides the same or more features > > >> than > > >> > Jekyll out the box: > > >> > - search! That is a big one, not available with Jekyll > > >> > - include of external files > > >> > - syntax highlighting > > >> > - plugins system > > >> > - custom theme > > >> > > > >> > Best. > > >> > > > >> > On Sat, 7 Oct 2017, 17:10 Mark McKenna, <m4rkmcke...@apache.org> > > wrote: > > >> > > > >> >> Thomas this looks really clean great work. > > >> >> > > >> >> How much work do you think it will take to maintain vs our current > > >> >> solution? > > >> >> What do you see as being the current feature gaps? > > >> >> > > >> >> M > > >> >> > > >> >> On Fri, 6 Oct 2017 at 14:55 Thomas Bouron < > > >> thomas.bou...@cloudsoftcorp.com > > >> >> wrote: > > >> >> > > >> >>> Hi Richard. > > >> >>> > > >> >>> Of course, I pushed it to my fork on the branch > > >> `experiment/gitbook`[1] > > >> >>> Glad you like it :) > > >> >>> > > >> >>> Best. > > >> >>> > > >> >>> [1] https://github.com/tbouron/brooklyn-docs/tree/experiment/ > > gitbook > > >> >>> > > >> >>> On Fri, 6 Oct 2017 at 13:53 Andrea Turli <and...@cloudsoft.io> > > wrote: > > >> >>> > > >> >>>> +1 Thomas, didn't know Gitbook at all (that's why I suggested > > >> >>> readthedocs) > > >> >>>> but looks pretty good! > > >> >>>> > > >> >>>> Il 06/ott/2017 15:37, "Richard Downer" <rich...@apache.org> ha > > >> >> scritto: > > >> >>>> Hi Thomas, > > >> >>>> > > >> >>>> I withdraw my previous comments - I looked at ReadTheDocs last > year > > >> and > > >> >>> was > > >> >>>> pessimistic, but it seems that GitBook this year is a different > > story > > >> >> :-) > > >> >>>> This is worth pursuing IMO. What did you need to do to get this > > >> >> working? > > >> >>>> Did you have to do any work on the brooklyn-docs source - if so > > could > > >> >> you > > >> >>>> share a link to your repo? > > >> >>>> > > >> >>>> Thanks > > >> >>>> Richard. > > >> >>>> > > >> >>>> > > >> >>>> On 6 October 2017 at 13:18, Thomas Bouron > > >> <thomas.bouron@cloudsoftcorp. > > >> >>> com > > >> >>>> wrote: > > >> >>>> > > >> >>>>> Hi All. > > >> >>>>> > > >> >>>>> A demo is worth a thousand words so here is a gitbook adaptation > > of > > >> >> our > > >> >>>>> current documentation[1] (and only documentation) > > >> >>>>> This took me only a couple of hours. There are still things to > > >> >>>>> fix/update/remove like unsupported liquid tags but for the most > > >> part, > > >> >>> it > > >> >>>>> works like a charm. > > >> >>>>> Search is available from the search field on the top left and > > >> PDF[2], > > >> >>>>> epub[3] amd mobi[4] versions are also available. > > >> >>>>> The build took only 10 sec + 10 more per offline version. > > >> >>>>> > > >> >>>>> The table of content mirrors exactly what we currently have, > > except > > >> >>> that > > >> >>>> I > > >> >>>>> have limited it to only 2 sub-levels. It means that some pages > are > > >> >>>> missing > > >> >>>>> but I think it demonstrates that our current menu organisation > > could > > >> >> be > > >> >>>>> vastly improved. > > >> >>>>> > > >> >>>>> Couple of thoughts on Alex's points: > > >> >>>>> > > >> >>>>>> * for the examples, import source code that is actually used in > > >> >> tests > > >> >>>>> (!!!) > > >> >>>>> > > >> >>>>> Indeed, an overhaul does not solve it, nor our current > framework. > > >> But > > >> >>>> both > > >> >>>>> can implement it. > > >> >>>>> > > >> >>>>>> * check links > > >> >>>>> Gitbook checks internal links at compile time and refuses to > build > > >> if > > >> >>>>> something is wrong. AFAIK, there is nothing in the Gitbook world > > to > > >> >>> check > > >> >>>>> the validity of external links like the Jekyll plugin does. > There > > >> are > > >> >>>>> probably external tools that we can integrate in our build > > pipeline > > >> >> to > > >> >>>>> cover this. However, it seems that even if we have this tool, we > > >> >> don't > > >> >>>> use > > >> >>>>> it when pushing the website (as I get a lot of errors locally) > > >> >>>>> Realistically, we will always have broken links, things move > > around > > >> >> all > > >> >>>> the > > >> >>>>> time. Checking external links is a nice-to-have but far from > > being a > > >> >>>>> perfect solution. In any case, I don't see this point as > important > > >> as > > >> >>> you > > >> >>>>> do. > > >> >>>>> > > >> >>>>>> * think through user flow > > >> >>>>> The clear Gitbook menu exposes this pretty well IMO and better > > >> >> compared > > >> >>>> to > > >> >>>>> the current version so that's a win. > > >> >>>>> > > >> >>>>> Best. > > >> >>>>> > > >> >>>>> [1] https://tbouron.github.io/brooklyn-docs/ > > >> >>>>> [2] https://tbouron.github.io/brooklyn-docs/brooklyn.pdf > > >> >>>>> [3] https://tbouron.github.io/brooklyn-docs/brooklyn.epub > > >> >>>>> [4] https://tbouron.github.io/brooklyn-docs/brooklyn.mobi > > >> >>>>> > > >> >>>>> > > >> >>>>> On Thu, 5 Oct 2017 at 12:47 Richard Downer <rich...@apache.org> > > >> >> wrote: > > >> >>>>>> Thank you for the research you have done Thomas. I've had > similar > > >> >>>>> thoughts > > >> >>>>>> myself. The original goal of our web+docs was to integrate them > > in > > >> >>> such > > >> >>>> a > > >> >>>>>> way that we had a versioned user guide that integrated > perfectly > > >> >> with > > >> >>>> the > > >> >>>>>> main website. At the time, Markdown tools were relatively > > immature, > > >> >>>> with > > >> >>>>>> Jekyll leading the pack (and being the fashionable choice), and > > >> >> very > > >> >>>>> little > > >> >>>>>> in the way of viable apps for generating books with structure > and > > >> >>>> tables > > >> >>>>> of > > >> >>>>>> contents. We did the best we could with the tools we had, but > > they > > >> >>>> needed > > >> >>>>>> significant extensions (via Jekyll plugins and build > scripting). > > >> >>> Those > > >> >>>>>> plugins and scripts have turned into something fairly hairy - > IMO > > >> >> we > > >> >>>>>> shouldn't need to have to write this much code[1] to generate a > > >> >>> static > > >> >>>>> site > > >> >>>>>> and manual. With hindsight, I would not have argued in favour > of > > >> >> this > > >> >>>>>> model. If I do write my book[2] I will most likely be writing > it > > in > > >> >>>>>> ReStructuredText and processing it with Sphinx (and no > additional > > >> >>>>>> scripting/tooling!). > > >> >>>>>> > > >> >>>>>> That said, when I have looked at changing Brooklyn's > > documentation > > >> >>>>> system, > > >> >>>>>> it has not looked easy. With our home-grown TOC generating > code, > > >> >>> we're > > >> >>>>> not > > >> >>>>>> off-the-shelf compatible with other systems. Moving to another > > >> >>> system, > > >> >>>>> even > > >> >>>>>> if it is Markdown-based, would still involve a lot of manual > work > > >> >>>>> changing > > >> >>>>>> our document metadata to the new system, and adapting to > replace > > >> >> the > > >> >>>>> Jekyll > > >> >>>>>> plugins and the content that uses them (e.g. syntax > highlighting, > > >> >>> file > > >> >>>>>> inclusion). Unless you have discovered something I didn't, > > Thomas, > > >> >>> then > > >> >>>> I > > >> >>>>>> fear this will be a lot of work, mostly manual. > > >> >>>>>> > > >> >>>>>> In short, yes I like the idea of replacing our home-grown and > > >> >>>>>> home-maintained code with an existing and supported app, but > no I > > >> >>> don't > > >> >>>>>> think the effort of a big-bang migration justifies the results > > *at > > >> >>> this > > >> >>>>>> time*. > > >> >>>>>> > > >> >>>>>> Some things I would support: > > >> >>>>>> > > >> >>>>>> - Continued incremental improvements to both the website and > the > > >> >> user > > >> >>>>>> guide. IMO we have more problems with the content than with the > > >> >>>> tooling, > > >> >>>>>> and we can still make a lot of improvements to the usability of > > our > > >> >>>> docs > > >> >>>>>> and website without tooling changes. > > >> >>>>>> > > >> >>>>>> - Breaking the tight integration between website and user > guide. > > >> >>> "Fork" > > >> >>>>> the > > >> >>>>>> existing infrastructure but then have two build systems > tailored > > >> >> for > > >> >>>>> their > > >> >>>>>> purpose rather than one that tried to meet two different needs. > > >> >> Would > > >> >>>>> allow > > >> >>>>>> the existing stuff to continue to work while opening the door > to > > >> >>>>> replacing > > >> >>>>>> the guide tooling and redeveloping the website, independently > of > > >> >> each > > >> >>>>>> other, at a future date. > > >> >>>>>> > > >> >>>>>> - Evaluating how other systems use metadata to describe the > book > > >> >>>>> structure, > > >> >>>>>> and gradually adding support for this to our own tools and > > >> >> migrating > > >> >>>>>> content. Then at a later date, when the content is > > >> >> nearly-compatible > > >> >>>> with > > >> >>>>>> GitBook or some other system, it'll be easier to do the > > migration. > > >> >>>>>> > > >> >>>>>> What do you think? Will following an incremental approach like > > this > > >> >>>> allow > > >> >>>>>> us to make improvements gradually rather than a "big bang" > > >> >>> replacement > > >> >>>> of > > >> >>>>>> tooling? > > >> >>>>>> > > >> >>>>>> Richard. > > >> >>>>>> > > >> >>>>>> [1] > > >> >> https://gist.github.com/rdowner/a09a268b37904a03c452797e7afe56ca > > >> >>>> but > > >> >>>>>> consider the COCOMO figures with appropriate cynicism > > >> >>>>>> [2] > > >> >>>>>> > > >> >>>>>> > > >> >> https://lists.apache.org/thread.html/ > 6f19475bbc0570a3b9e3d1ae1b75b2 > > >> >>>>> b8ee4b2485b3b41d085c342dff@%3Cdev.brooklyn.apache.org%3E > > >> >>>>>> On 5 October 2017 at 11:23, Thomas Bouron > > >> >>> <thomas.bouron@cloudsoftcorp. > > >> >>>>> com > > >> >>>>>> wrote: > > >> >>>>>> > > >> >>>>>>> Hi all. > > >> >>>>>>> > > >> >>>>>>> It's been a couple of weeks that I started to look at how to > > >> >>> improve > > >> >>>>> and > > >> >>>>>>> simplify the Brookyln website[1]. As I said on the Brooklyn > 1.0 > > >> >>>>>> thread[2], > > >> >>>>>>> I think we need to sort this out before releasing 1.0. > > >> >>>>>>> > > >> >>>>>>> I have looked for a framework / library to handle both the > > >> >> website > > >> >>>> and > > >> >>>>>>> documentation the same way we do it right now. To determine > what > > >> >>> was > > >> >>>>> the > > >> >>>>>>> best fit, I based my analysis on the following criteria: > > >> >>>>>>> - Able to take markdown files and generate HTML from them. > > >> >>>>>>> - Keep the folder structure intact (currently, pages that > seems > > >> >> in > > >> >>>> the > > >> >>>>>> same > > >> >>>>>>> logical group - take pages in the download section[3] menu - > > jump > > >> >>>> into > > >> >>>>> a > > >> >>>>>>> different folder/category/section which is very confusing) > > >> >>>>>>> - Be skinnable > > >> >>>>>>> - Able to handle versions for documentation. > > >> >>>>>>> - Able to generate PDF version of documentation. > > >> >>>>>>> - Be as "stock" as possible to limit maintenance and pain > during > > >> >>>>> upgrade > > >> >>>>>>> (our current website still uses Jekyll 2.x). > > >> >>>>>>> > > >> >>>>>>> 2 contenders clearly jumped out from this: > > >> >>>>>>> - Jekyll[4] > > >> >>>>>>> - Gitbook[5] > > >> >>>>>>> > > >> >>>>>>> ---- > > >> >>>>>>> Jekyll > > >> >>>>>>> > > >> >>>>>>> With the version 3, Jekyll now has a concept of collections[6] > > >> >>> which > > >> >>>>> can > > >> >>>>>>> generate pages from markdown files and keep the folder > > structure. > > >> >>>>>>> The menu can be generated based on this folder structure (with > > >> >>> depth > > >> >>>>>>> limitation for example) in combination of some clever liquid > > tags > > >> >>> and > > >> >>>>>>> `include`. However, it will be hard to control the order of > > items > > >> >>>>>> appearing > > >> >>>>>>> on the menu. Another easy solution would be maintain list of > > >> >> links > > >> >>>> for > > >> >>>>>> the > > >> >>>>>>> menu to be generated. > > >> >>>>>>> There are plugins to generate PDF[7], which happens during > > >> >> compile > > >> >>>>> time. > > >> >>>>>>> Finally, Jekyll is highly skinnable with built-in or custom > > >> >> themes. > > >> >>>>>>> ---- > > >> >>>>>>> Gitbook > > >> >>>>>>> > > >> >>>>>>> Gitbook, in its open source version, handles out of the box > doc > > >> >>>>>> versioning, > > >> >>>>>>> PDF generation at runtime (so it seems) HTML pages generation > > >> >> from > > >> >>>>>>> markdown. The menu is built-in feature, based on a simple > > >> >> markdown > > >> >>>> list > > >> >>>>>> of > > >> >>>>>>> links[8]. This means we need to maintain it but there is a > good > > >> >>>> chance > > >> >>>>> we > > >> >>>>>>> will have to do this with Jekyll as well. Finally, Gitbook is > > >> >> also > > >> >>>>> easily > > >> >>>>>>> skinnable[9]. > > >> >>>>>>> > > >> >>>>>>> ---- > > >> >>>>>>> Both frameworks offer mostly the same features. However, > Jekyll > > >> >> is > > >> >>>>> easier > > >> >>>>>>> to build a website that looks like a "corporate" one whereas > > with > > >> >>>>>> Gitbook, > > >> >>>>>>> you are "stuck" with the design principals it was created, > i.e. > > >> >>> serve > > >> >>>>>>> documentation only. But for this very purpose, it is extremely > > >> >> good > > >> >>>> and > > >> >>>>>>> easy. > > >> >>>>>>> > > >> >>>>>>> Our website is the combination of both a "corporate website" > > >> >> (i.e. > > >> >>>>> about, > > >> >>>>>>> getting started, community, etc - few pages that describe the > > >> >>>> project) > > >> >>>>>> and > > >> >>>>>>> a documentation. > > >> >>>>>>> > > >> >>>>>>> Which leads me to my proposal: separate the website from the > > >> >>>>>> documentation, > > >> >>>>>>> at least in terms of how we build it. What I mean by this is: > > >> >>>>>>> - Use Jekyll (or even nothing) for the website, except the > > >> >>>>> documentation > > >> >>>>>>> part. This will let us build a nice theme (based on Bootstrap > 4 > > >> >> for > > >> >>>>>>> example) without to worry about complicated plugins and custom > > >> >> code > > >> >>>> for > > >> >>>>>> the > > >> >>>>>>> documentation. > > >> >>>>>>> - Use Gitbook for the documentation alone, applying/adapting > the > > >> >>>> theme > > >> >>>>> we > > >> >>>>>>> will create from the point above. > > >> >>>>>>> > > >> >>>>>>> Best. > > >> >>>>>>> > > >> >>>>>>> [1] https://brooklyn.apache.org/ > > >> >>>>>>> [2] > > >> >>>>>>> https://lists.apache.org/thread.html/ > > >> >>> dae4468aa7ef77af9dc8aca24b8434 > > >> >>>>>>> e9782efbd50fa876618cccf980@%3Cdev.brooklyn.apache.org%3E > > >> >>>>>>> [3] https://brooklyn.apache.org/download/index.html > > >> >>>>>>> [4] https://jekyllrb.com/ > > >> >>>>>>> [5] https://github.com/GitbookIO/gitbook > > >> >>>>>>> [6] https://jekyllrb.com/docs/collections/ > > >> >>>>>>> [7] http://abemedia.co.uk/jekyll-pdf/ > > >> >>>>>>> [8] https://toolchain.gitbook.com/pages.html > > >> >>>>>>> [9] https://toolchain.gitbook.com/themes/ > > >> >>>>>>> -- > > >> >>>>>>> > > >> >>>>>>> Thomas Bouron • Senior Software Engineer @ Cloudsoft > Corporation > > >> >> • > > >> >>>>>>> https://cloudsoft.io/ > > >> >>>>>>> Github: https://github.com/tbouron > > >> >>>>>>> Twitter: https://twitter.com/eltibouron > > >> >>>>>>> > > >> >>>>> -- > > >> >>>>> > > >> >>>>> Thomas Bouron • Senior Software Engineer @ Cloudsoft > Corporation • > > >> >>>>> https://cloudsoft.io/ > > >> >>>>> Github: https://github.com/tbouron > > >> >>>>> Twitter: https://twitter.com/eltibouron > > >> >>>>> > > >> >>> -- > > >> >>> > > >> >>> Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > > >> >>> https://cloudsoft.io/ > > >> >>> Github: https://github.com/tbouron > > >> >>> Twitter: https://twitter.com/eltibouron > > >> >>> > > >> > > >> -- > > > > > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > > > https://cloudsoft.io/ > > > Github: https://github.com/tbouron > > > Twitter: https://twitter.com/eltibouron > > > > > -- > > > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > > https://cloudsoft.io/ > > Github: https://github.com/tbouron > > Twitter: https://twitter.com/eltibouron > > >
