Hello Apexers, I'd like to propose a process for creating, storing, updating, and displaying documentation of the apex and malhar projects.
*## Goals* The goals for docs that I had in mind are: - Versioned with the code base - Easy to contribute to by anyone - Displayed on apex.incubator.apache.org - Simple deploy process - Indexable by search engines - Easily navigable; table of contents, support for nested "groups" of docs, responsive search *## Proposal* My proposal is as follows: 1. Source files for docs will live in their respective git repositories, under a `docs` folder. Format for said files will be github-flavored markdown <https://help.github.com/articles/github-flavored-markdown/> (In the future, other formats could be supported). This takes care of the first two goals listed above. 2. In addition to the docs themselves, a single json file acting as an index for the rendered docs will exist in `docs/index.json`. This will be used to render a left navigation bar in the docs section. 3. Create a gulp <http://gulpjs.com/> task which: a. pulls down these files for each released version of apex and malhar (using the Github API) b. creates HTML versions of each project/version/doc, each with the same left-side navigation c. creates a JSON file (per version) containing the raw markdown content that can be used by Javascript on the page to power a fast search function in the left-side navigation. 4. As part of the release process, run the aforementioned gulp task and push to the site using the current `build.sh` script *## Mockups* This would result in an index page on *apex.incubator.apache.org/docs/index.html <http://apex.incubator.apache.org/docs/index.html>*: [image: Inline image 1] And this is a mockup of looking at an individual doc, e.g. *apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html <http://apex.incubator.apache.org/docs/v3.2.0-incubating/contributing.html>* : [image: Inline image 2] The search feature could filter the docs shown in the side bar in real-time: [image: Inline image 3] *## Caveats* No plan is perfect, and this one is no exception. A few caveats to this plan are: - Any updates to the docs in the repos require a manual build and push to the website. - This would be a custom solution, and while I will rely on existing modules as much as possible, there will be a little glue-code - The JSON files used by the search feature may become very large if the docs become extremely large. This may hurt browser performance. I doubt this will ever be a problem though. *## Alternatives* We could approach the problem of docs in a few different ways. Most notably are solutions hosted by others (i.e., Documentation as a Service). Some examples of this solution: Atlassian Confluence <https://developer.atlassian.com/opensource/>, Readme.io, and Readthedocs <https://readthedocs.org/>. I personally take issue with these for the following reasons: - docs not versioned along with the code (readthedocs is the exception here) - most solutions are not very user-friendly (no real-time search) - css-skinning is difficult or impossible - system exists outside of current projects/website If the community rejects the proposal put forth here, my choice of alternative would be ReadTheDocs (see the features <http://read-the-docs.readthedocs.org/en/latest/features.html>). *Pros of ReadTheDocs:* - docs are also versioned with codebase in git repo - git hook automatically updates docs without any manual step - out-of-the-box support for showing different versions of the docs - decent default theme *Cons of ReadTheDocs are: * - separately hosted - learning curve for usage - difficulty for skinning - difficulty extending/adding functionality - Slow search/frustrating left-hand navigation *## Conclusions* Documentation is one of the first places potential new community members/developers will go when vetting Apex for use. Ideally, it should dazzle and inspire. I think the status-quo for documentation on open-source projects is a low bar, and I am willing to put in the bit of work to lay the foundation for excellent docs. Again, if my custom proposal seems too problematic, my vote is for using ReadTheDocs. Please share your thoughts. -Andy -- Regards, Andy Perlitch Software Engineer DataTorrent Inc (408)829-9319
