All, As a co-conspirator on this I am obviously +1 to it :-) But it would be good to hear some more POVs on this.
Thomas and I (mostly Thomas) have come up with a solution which we believe has "feature parity" with the current docs solution, except for some cases where we believe the feature is not significant (or actually an anti-feature). This solution provides something which is easier to navigate, easier to turn into PDF, and it does all these using off-the-shelf tools and no bespoke code (no more Ruby-based Jekyll plugins!) The new solution does look different, in that it is not a slave to our current website design, but its styling does incorporate the key design elements (typography, colour scheme). It is not tightly integrated with the website as the current docs is, but we do not consider this to be a major issue. In my personal opinion, having such tight integration turned into an anti-feature, as it caused conflicts between the version-related and non-version-related pages. (For an example, browse for earlier versions of the Brooklyn documentation - suddenly there are changes to the website styling and the illusion of close integration is lost). I am in favour of proceeding with this change and seeing how it goes - if it does not work out, then we can revert back. Any documentation changes in the affected period should be easily backportable to the old-style website. In addition to Thomas's branch which forks the existing brooklyn-docs and deletes the website, I have an alternative fork which deletes the docs, where work on the website-only tools and content can be done. With the docs removed there's a lot of scope to remove lots of code from the tooling. In the future we would maybe want to look at further changes to the website, such as upgrading to a newer version of Jekyll and investigating newer Jekyll plugins, with the aim of removing all our bespoke code. Richard. On 15 October 2017 at 17:20, Thomas Bouron <thomas.bou...@cloudsoftcorp.com> wrote: > Hi all. > > I just pushed the last set of commits to my fork for the docs[1]. This now > contains the docs generated by gitbook and a build script[2] to generate > the javadoc at the right place (i.e. misc/javadoc). The last commit[3] > updates the README to include the updated release instructions. With this, > I think we are now ready to go. > > In terms of next steps, Richard suggested to have the codebase for website > and docs in 2 separate branches. This is how GitHub pages works for example > and it makes sense to me. Unless someone is against this, I will create 2 > branches on the git repo, `website` and `docs` and open a PR again `docs` > on Monday. > > Best. > > [1] https://github.com/tbouron/brooklyn-docs/tree/experiment/gitbook > [2] > https://github.com/tbouron/brooklyn-docs/blob/experiment/ > gitbook/javadoc/build.sh > [3] > https://github.com/tbouron/brooklyn-docs/commit/ > e931560888ce51625e56e6202ead757f9d876090 > > On Fri, 13 Oct 2017 at 13:02 Thomas Bouron <thomas.bouron@cloudsoftcorp. > com> > wrote: > > > Hi Alex, all. > > > > For the past few days, I worked with Richard on this doc spike. Here is a > > short summary of what we have done: > > - add collapse/expandable sections on the left menu > > - add documentation versioning (version dropdown, top left) > > - improve PDF output > > - fix all internal links > > - use brooklyn styling > > > > One thing left is to use YAML examples from the Brooklyn codebase. There > > is a plugin to do exactly that (which supports to import only a snippet > > from a file) > > Unfortunately, most of the doc examples exist only in the docs repo > > therefore I think we should resolve this later on, as an incremental > change. > > > > I published the new version on the same URL as before[1], hope you will > > like it. Spoiler alert: I do :) > > > > PS: I created a gitbook plugin[2] to auto-generate menus on pages like > > this one[3]. However, I had a chat with a maintainer (founder?) of > Gitbook > > who told me[4] that version 4 will include, out of the box, most plugins > we > > currently use! They have a preview[5] of what is does. > > > > Best. > > > > [1] https://tbouron.github.io/brooklyn-docs > > [2] https://github.com/tbouron/gitbook-plugin-partial-summary > > [3] https://tbouron.github.io/brooklyn-docs/start/ > > [4] > > https://gitbook-community.slack.com/archives/C0B5XG0GK/p1507893620000134 > > [5] https://betadocs.gitbook.com/features > > > > On Tue, 10 Oct 2017 at 12:20 Alex Heneveld < > > alex.henev...@cloudsoftcorp.com> wrote: > > > >> > >> Thomas- > >> > >> Had a deeper look -- gitbook has moved things forward a lot. Sounds like > >> it will let us throw away a lot of our home-grown docs-building and > >> toc-building code and have good search. Look forward to seeing how it > >> shapes up with styling and guide-v-website integration. > >> > >> Best > >> Alex > >> > >> > >> On 09/10/2017 09:54, Thomas Bouron wrote: > >> > Thanks Mark. > >> > > >> > Regarding maintenance, it will be as easy as the current version. > >> Updating > >> > docs means updating markdown files. Adding/moving pages requires to > >> modify > >> > the `SUMMARY.md` but that's it. > >> > One really cool thing is that Gitbook is a node app: really simple to > >> > install/run compare to our current solution which runs only on an old > >> > version of ruby => no more pain of using different versions of ruby on > >> your > >> > environment. > >> > > >> > In terms of feature gaps, Gitbook provides the same or more features > >> than > >> > Jekyll out the box: > >> > - search! That is a big one, not available with Jekyll > >> > - include of external files > >> > - syntax highlighting > >> > - plugins system > >> > - custom theme > >> > > >> > Best. > >> > > >> > On Sat, 7 Oct 2017, 17:10 Mark McKenna, <m4rkmcke...@apache.org> > wrote: > >> > > >> >> Thomas this looks really clean great work. > >> >> > >> >> How much work do you think it will take to maintain vs our current > >> >> solution? > >> >> What do you see as being the current feature gaps? > >> >> > >> >> M > >> >> > >> >> On Fri, 6 Oct 2017 at 14:55 Thomas Bouron < > >> thomas.bou...@cloudsoftcorp.com > >> >> wrote: > >> >> > >> >>> Hi Richard. > >> >>> > >> >>> Of course, I pushed it to my fork on the branch > >> `experiment/gitbook`[1] > >> >>> Glad you like it :) > >> >>> > >> >>> Best. > >> >>> > >> >>> [1] https://github.com/tbouron/brooklyn-docs/tree/experiment/ > gitbook > >> >>> > >> >>> On Fri, 6 Oct 2017 at 13:53 Andrea Turli <and...@cloudsoft.io> > wrote: > >> >>> > >> >>>> +1 Thomas, didn't know Gitbook at all (that's why I suggested > >> >>> readthedocs) > >> >>>> but looks pretty good! > >> >>>> > >> >>>> Il 06/ott/2017 15:37, "Richard Downer" <rich...@apache.org> ha > >> >> scritto: > >> >>>> Hi Thomas, > >> >>>> > >> >>>> I withdraw my previous comments - I looked at ReadTheDocs last year > >> and > >> >>> was > >> >>>> pessimistic, but it seems that GitBook this year is a different > story > >> >> :-) > >> >>>> This is worth pursuing IMO. What did you need to do to get this > >> >> working? > >> >>>> Did you have to do any work on the brooklyn-docs source - if so > could > >> >> you > >> >>>> share a link to your repo? > >> >>>> > >> >>>> Thanks > >> >>>> Richard. > >> >>>> > >> >>>> > >> >>>> On 6 October 2017 at 13:18, Thomas Bouron > >> <thomas.bouron@cloudsoftcorp. > >> >>> com > >> >>>> wrote: > >> >>>> > >> >>>>> Hi All. > >> >>>>> > >> >>>>> A demo is worth a thousand words so here is a gitbook adaptation > of > >> >> our > >> >>>>> current documentation[1] (and only documentation) > >> >>>>> This took me only a couple of hours. There are still things to > >> >>>>> fix/update/remove like unsupported liquid tags but for the most > >> part, > >> >>> it > >> >>>>> works like a charm. > >> >>>>> Search is available from the search field on the top left and > >> PDF[2], > >> >>>>> epub[3] amd mobi[4] versions are also available. > >> >>>>> The build took only 10 sec + 10 more per offline version. > >> >>>>> > >> >>>>> The table of content mirrors exactly what we currently have, > except > >> >>> that > >> >>>> I > >> >>>>> have limited it to only 2 sub-levels. It means that some pages are > >> >>>> missing > >> >>>>> but I think it demonstrates that our current menu organisation > could > >> >> be > >> >>>>> vastly improved. > >> >>>>> > >> >>>>> Couple of thoughts on Alex's points: > >> >>>>> > >> >>>>>> * for the examples, import source code that is actually used in > >> >> tests > >> >>>>> (!!!) > >> >>>>> > >> >>>>> Indeed, an overhaul does not solve it, nor our current framework. > >> But > >> >>>> both > >> >>>>> can implement it. > >> >>>>> > >> >>>>>> * check links > >> >>>>> Gitbook checks internal links at compile time and refuses to build > >> if > >> >>>>> something is wrong. AFAIK, there is nothing in the Gitbook world > to > >> >>> check > >> >>>>> the validity of external links like the Jekyll plugin does. There > >> are > >> >>>>> probably external tools that we can integrate in our build > pipeline > >> >> to > >> >>>>> cover this. However, it seems that even if we have this tool, we > >> >> don't > >> >>>> use > >> >>>>> it when pushing the website (as I get a lot of errors locally) > >> >>>>> Realistically, we will always have broken links, things move > around > >> >> all > >> >>>> the > >> >>>>> time. Checking external links is a nice-to-have but far from > being a > >> >>>>> perfect solution. In any case, I don't see this point as important > >> as > >> >>> you > >> >>>>> do. > >> >>>>> > >> >>>>>> * think through user flow > >> >>>>> The clear Gitbook menu exposes this pretty well IMO and better > >> >> compared > >> >>>> to > >> >>>>> the current version so that's a win. > >> >>>>> > >> >>>>> Best. > >> >>>>> > >> >>>>> [1] https://tbouron.github.io/brooklyn-docs/ > >> >>>>> [2] https://tbouron.github.io/brooklyn-docs/brooklyn.pdf > >> >>>>> [3] https://tbouron.github.io/brooklyn-docs/brooklyn.epub > >> >>>>> [4] https://tbouron.github.io/brooklyn-docs/brooklyn.mobi > >> >>>>> > >> >>>>> > >> >>>>> On Thu, 5 Oct 2017 at 12:47 Richard Downer <rich...@apache.org> > >> >> wrote: > >> >>>>>> Thank you for the research you have done Thomas. I've had similar > >> >>>>> thoughts > >> >>>>>> myself. The original goal of our web+docs was to integrate them > in > >> >>> such > >> >>>> a > >> >>>>>> way that we had a versioned user guide that integrated perfectly > >> >> with > >> >>>> the > >> >>>>>> main website. At the time, Markdown tools were relatively > immature, > >> >>>> with > >> >>>>>> Jekyll leading the pack (and being the fashionable choice), and > >> >> very > >> >>>>> little > >> >>>>>> in the way of viable apps for generating books with structure and > >> >>>> tables > >> >>>>> of > >> >>>>>> contents. We did the best we could with the tools we had, but > they > >> >>>> needed > >> >>>>>> significant extensions (via Jekyll plugins and build scripting). > >> >>> Those > >> >>>>>> plugins and scripts have turned into something fairly hairy - IMO > >> >> we > >> >>>>>> shouldn't need to have to write this much code[1] to generate a > >> >>> static > >> >>>>> site > >> >>>>>> and manual. With hindsight, I would not have argued in favour of > >> >> this > >> >>>>>> model. If I do write my book[2] I will most likely be writing it > in > >> >>>>>> ReStructuredText and processing it with Sphinx (and no additional > >> >>>>>> scripting/tooling!). > >> >>>>>> > >> >>>>>> That said, when I have looked at changing Brooklyn's > documentation > >> >>>>> system, > >> >>>>>> it has not looked easy. With our home-grown TOC generating code, > >> >>> we're > >> >>>>> not > >> >>>>>> off-the-shelf compatible with other systems. Moving to another > >> >>> system, > >> >>>>> even > >> >>>>>> if it is Markdown-based, would still involve a lot of manual work > >> >>>>> changing > >> >>>>>> our document metadata to the new system, and adapting to replace > >> >> the > >> >>>>> Jekyll > >> >>>>>> plugins and the content that uses them (e.g. syntax highlighting, > >> >>> file > >> >>>>>> inclusion). Unless you have discovered something I didn't, > Thomas, > >> >>> then > >> >>>> I > >> >>>>>> fear this will be a lot of work, mostly manual. > >> >>>>>> > >> >>>>>> In short, yes I like the idea of replacing our home-grown and > >> >>>>>> home-maintained code with an existing and supported app, but no I > >> >>> don't > >> >>>>>> think the effort of a big-bang migration justifies the results > *at > >> >>> this > >> >>>>>> time*. > >> >>>>>> > >> >>>>>> Some things I would support: > >> >>>>>> > >> >>>>>> - Continued incremental improvements to both the website and the > >> >> user > >> >>>>>> guide. IMO we have more problems with the content than with the > >> >>>> tooling, > >> >>>>>> and we can still make a lot of improvements to the usability of > our > >> >>>> docs > >> >>>>>> and website without tooling changes. > >> >>>>>> > >> >>>>>> - Breaking the tight integration between website and user guide. > >> >>> "Fork" > >> >>>>> the > >> >>>>>> existing infrastructure but then have two build systems tailored > >> >> for > >> >>>>> their > >> >>>>>> purpose rather than one that tried to meet two different needs. > >> >> Would > >> >>>>> allow > >> >>>>>> the existing stuff to continue to work while opening the door to > >> >>>>> replacing > >> >>>>>> the guide tooling and redeveloping the website, independently of > >> >> each > >> >>>>>> other, at a future date. > >> >>>>>> > >> >>>>>> - Evaluating how other systems use metadata to describe the book > >> >>>>> structure, > >> >>>>>> and gradually adding support for this to our own tools and > >> >> migrating > >> >>>>>> content. Then at a later date, when the content is > >> >> nearly-compatible > >> >>>> with > >> >>>>>> GitBook or some other system, it'll be easier to do the > migration. > >> >>>>>> > >> >>>>>> What do you think? Will following an incremental approach like > this > >> >>>> allow > >> >>>>>> us to make improvements gradually rather than a "big bang" > >> >>> replacement > >> >>>> of > >> >>>>>> tooling? > >> >>>>>> > >> >>>>>> Richard. > >> >>>>>> > >> >>>>>> [1] > >> >> https://gist.github.com/rdowner/a09a268b37904a03c452797e7afe56ca > >> >>>> but > >> >>>>>> consider the COCOMO figures with appropriate cynicism > >> >>>>>> [2] > >> >>>>>> > >> >>>>>> > >> >> https://lists.apache.org/thread.html/6f19475bbc0570a3b9e3d1ae1b75b2 > >> >>>>> b8ee4b2485b3b41d085c342dff@%3Cdev.brooklyn.apache.org%3E > >> >>>>>> On 5 October 2017 at 11:23, Thomas Bouron > >> >>> <thomas.bouron@cloudsoftcorp. > >> >>>>> com > >> >>>>>> wrote: > >> >>>>>> > >> >>>>>>> Hi all. > >> >>>>>>> > >> >>>>>>> It's been a couple of weeks that I started to look at how to > >> >>> improve > >> >>>>> and > >> >>>>>>> simplify the Brookyln website[1]. As I said on the Brooklyn 1.0 > >> >>>>>> thread[2], > >> >>>>>>> I think we need to sort this out before releasing 1.0. > >> >>>>>>> > >> >>>>>>> I have looked for a framework / library to handle both the > >> >> website > >> >>>> and > >> >>>>>>> documentation the same way we do it right now. To determine what > >> >>> was > >> >>>>> the > >> >>>>>>> best fit, I based my analysis on the following criteria: > >> >>>>>>> - Able to take markdown files and generate HTML from them. > >> >>>>>>> - Keep the folder structure intact (currently, pages that seems > >> >> in > >> >>>> the > >> >>>>>> same > >> >>>>>>> logical group - take pages in the download section[3] menu - > jump > >> >>>> into > >> >>>>> a > >> >>>>>>> different folder/category/section which is very confusing) > >> >>>>>>> - Be skinnable > >> >>>>>>> - Able to handle versions for documentation. > >> >>>>>>> - Able to generate PDF version of documentation. > >> >>>>>>> - Be as "stock" as possible to limit maintenance and pain during > >> >>>>> upgrade > >> >>>>>>> (our current website still uses Jekyll 2.x). > >> >>>>>>> > >> >>>>>>> 2 contenders clearly jumped out from this: > >> >>>>>>> - Jekyll[4] > >> >>>>>>> - Gitbook[5] > >> >>>>>>> > >> >>>>>>> ---- > >> >>>>>>> Jekyll > >> >>>>>>> > >> >>>>>>> With the version 3, Jekyll now has a concept of collections[6] > >> >>> which > >> >>>>> can > >> >>>>>>> generate pages from markdown files and keep the folder > structure. > >> >>>>>>> The menu can be generated based on this folder structure (with > >> >>> depth > >> >>>>>>> limitation for example) in combination of some clever liquid > tags > >> >>> and > >> >>>>>>> `include`. However, it will be hard to control the order of > items > >> >>>>>> appearing > >> >>>>>>> on the menu. Another easy solution would be maintain list of > >> >> links > >> >>>> for > >> >>>>>> the > >> >>>>>>> menu to be generated. > >> >>>>>>> There are plugins to generate PDF[7], which happens during > >> >> compile > >> >>>>> time. > >> >>>>>>> Finally, Jekyll is highly skinnable with built-in or custom > >> >> themes. > >> >>>>>>> ---- > >> >>>>>>> Gitbook > >> >>>>>>> > >> >>>>>>> Gitbook, in its open source version, handles out of the box doc > >> >>>>>> versioning, > >> >>>>>>> PDF generation at runtime (so it seems) HTML pages generation > >> >> from > >> >>>>>>> markdown. The menu is built-in feature, based on a simple > >> >> markdown > >> >>>> list > >> >>>>>> of > >> >>>>>>> links[8]. This means we need to maintain it but there is a good > >> >>>> chance > >> >>>>> we > >> >>>>>>> will have to do this with Jekyll as well. Finally, Gitbook is > >> >> also > >> >>>>> easily > >> >>>>>>> skinnable[9]. > >> >>>>>>> > >> >>>>>>> ---- > >> >>>>>>> Both frameworks offer mostly the same features. However, Jekyll > >> >> is > >> >>>>> easier > >> >>>>>>> to build a website that looks like a "corporate" one whereas > with > >> >>>>>> Gitbook, > >> >>>>>>> you are "stuck" with the design principals it was created, i.e. > >> >>> serve > >> >>>>>>> documentation only. But for this very purpose, it is extremely > >> >> good > >> >>>> and > >> >>>>>>> easy. > >> >>>>>>> > >> >>>>>>> Our website is the combination of both a "corporate website" > >> >> (i.e. > >> >>>>> about, > >> >>>>>>> getting started, community, etc - few pages that describe the > >> >>>> project) > >> >>>>>> and > >> >>>>>>> a documentation. > >> >>>>>>> > >> >>>>>>> Which leads me to my proposal: separate the website from the > >> >>>>>> documentation, > >> >>>>>>> at least in terms of how we build it. What I mean by this is: > >> >>>>>>> - Use Jekyll (or even nothing) for the website, except the > >> >>>>> documentation > >> >>>>>>> part. This will let us build a nice theme (based on Bootstrap 4 > >> >> for > >> >>>>>>> example) without to worry about complicated plugins and custom > >> >> code > >> >>>> for > >> >>>>>> the > >> >>>>>>> documentation. > >> >>>>>>> - Use Gitbook for the documentation alone, applying/adapting the > >> >>>> theme > >> >>>>> we > >> >>>>>>> will create from the point above. > >> >>>>>>> > >> >>>>>>> Best. > >> >>>>>>> > >> >>>>>>> [1] https://brooklyn.apache.org/ > >> >>>>>>> [2] > >> >>>>>>> https://lists.apache.org/thread.html/ > >> >>> dae4468aa7ef77af9dc8aca24b8434 > >> >>>>>>> e9782efbd50fa876618cccf980@%3Cdev.brooklyn.apache.org%3E > >> >>>>>>> [3] https://brooklyn.apache.org/download/index.html > >> >>>>>>> [4] https://jekyllrb.com/ > >> >>>>>>> [5] https://github.com/GitbookIO/gitbook > >> >>>>>>> [6] https://jekyllrb.com/docs/collections/ > >> >>>>>>> [7] http://abemedia.co.uk/jekyll-pdf/ > >> >>>>>>> [8] https://toolchain.gitbook.com/pages.html > >> >>>>>>> [9] https://toolchain.gitbook.com/themes/ > >> >>>>>>> -- > >> >>>>>>> > >> >>>>>>> Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation > >> >> • > >> >>>>>>> https://cloudsoft.io/ > >> >>>>>>> Github: https://github.com/tbouron > >> >>>>>>> Twitter: https://twitter.com/eltibouron > >> >>>>>>> > >> >>>>> -- > >> >>>>> > >> >>>>> Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > >> >>>>> https://cloudsoft.io/ > >> >>>>> Github: https://github.com/tbouron > >> >>>>> Twitter: https://twitter.com/eltibouron > >> >>>>> > >> >>> -- > >> >>> > >> >>> Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > >> >>> https://cloudsoft.io/ > >> >>> Github: https://github.com/tbouron > >> >>> Twitter: https://twitter.com/eltibouron > >> >>> > >> > >> -- > > > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > > https://cloudsoft.io/ > > Github: https://github.com/tbouron > > Twitter: https://twitter.com/eltibouron > > > -- > > Thomas Bouron • Senior Software Engineer @ Cloudsoft Corporation • > https://cloudsoft.io/ > Github: https://github.com/tbouron > Twitter: https://twitter.com/eltibouron >
