A couple more thoughts regarding the background on the CMS... the bulk of the focus of the CMS up until now has been to support really big sites like maven and openoffice, both of which have websites well over 5GB worth of text files. The build system was hollowed out and revamped to support external builds in other programming languages, as well as scaling up the perl builds to provide about 8 child processes that build the site in parallel by default. Full site build times for openoffice went from over 30 minutes to between 5 to 10 minutes depending on disk activity, which made experimentation with site-wide changes feasible on a reasonable timeframe. At first the project was in the habit of running the builds locally before checking in their changes because the delay in feedback post-commit was unreasonable, but with smart use of SSI the were able to cut down on the number of changes that required full site builds and hence no longer bother with the whole pre-build mumbo jumbo before checking in changes. The CMS is designed to only build pages that were altered, and if you just edit a page or two only those pages and the pages that depend on them will be built.
Now that I've turned my attention to thrift, there are several aspects that are better off being retuned based on the modest size of the site. First of all you have a sitemap url that lists all of your markdown-based pages, which would be a nonstarter for a site like openoffice. The implications of the sitemap url are that every page needs to be built in memory just to generate that page, which leads to a lot of redundancy in a build system designed to run as forked child processes. We can do better... I've done two things to make this work well- first I made the number of child processes "aka runners" tunable on a per-project basis, and since the sitemap is going to build every page anyway as the slowest page to build, I memoized the main view that builds the markdown pages for thrift. The best results are when there's only one runner so we get the full benefit of memoization: typical site build times are consistently under 2 seconds now. You'll appreciate why I am such a performance stickler for the CMS when you start using the bookmarklet, since builds are the only asynchronous part of the process from making a change to a page and publishing the results. Ideally when the system is working well the builds happen before you have time to go looking for them, but you do need to be mindful of not publishing before the built pages have been committed back to svn by buildbot. > On Monday, March 17, 2014 11:59 PM, Joe Schaefer <joe_schae...@yahoo.com> > wrote: > > Ok there's enough of an idea about how to port the rest of the site's > pages over based on the porting work I've already done that it's time > for me to step back and let you guys catch up. Basically the idea is that > while > there is templating support in the markdown sources, it's fairly limited > compared to what nanoc offers, but instead of writing a lot of complex > template > code into your sources, with the cms you just add code to lib/view.pm and/or > lib/path.pm to keep the document sources simple and just add more template > variables. What I just finished in lib/view.pm and lib/path.pm is directory > listing support for /docs.html but it can be carried over to similar pages. > > > > >> On Monday, March 17, 2014 9:16 PM, Joe Schaefer > <joe_schae...@yahoo.com> wrote: >> > Interesting ideas, thanks. FWIW I think that sort >> of thing might be best supported as a periodic cron >> if we can write something stable enough, because pulling >> stuff out of the git repo is something the CMS isn't going >> to do all that well being an svn application. >> >> The CMS build results are available at http://thrift.staging.apache.org/ >> feel free to play with the sources in repos/asf/thrift/cms-site/trunk. >> It's very fast, probably an order of magnitude faster than your current > >> tech. >> Whole site builds in two seconds; fractions of a second for typical mods to > >> content/ markdown pages. >> >> The CMS bookmarklet is compatible with the staging site FWIW. >> >> >> >> >>> On Monday, March 17, 2014 7:40 PM, Roger Meier >> <ro...@bufferoverflow.ch> wrote: >>> > Hi Joe & Co >>> >>> thanks for this test run with Apache CMS! >>> >>> What I would love to see is this: >>> >>> Source file: => Web Site URL: >>> test/README.md => thrift.apache.org/test >>> test/STATUS.md => thrift.apache.org/test/status >>> test/keys/README.md => thrift.apache.org/test/keys >>> lib/<lang>/README.md => thrift.apache.org/lib/<lang> >>> lib/<lang>/test/README.md => >> thrift.apache.org/lib/<lang>/test >>> lib/<lang>/examples/README.md => >>> thrift.apache.org/lib/<lang>/examples >>> compiler/cpp/README.md => thrift.apache.org/compiler >>> debian/README.md => thrift.apache.org/debian >>> contrib/<topic>/README.md => thrift.apache.org/<topic> >>> >>> just made the following patch to rename all file to .md within our > source >>> tree: https://issues.apache.org/jira/browse/THRIFT-2407 >>> ... we have *41* md's within the source tree and that's the > place >> where >>> people look for documentation first. => simple to manage >>> Would be user friendly to have all of this online with each release >>> with the URL layout mentioned above. >>> >>> just received the buildboot... >>> Do you have kind of a preview of your prototype? >>> >>> all the best! >>> -roger >>> ;-r >>> >>> >>> Quoting Jake Farrell <jfarr...@apache.org>: >>> >>>> No objections from me >>>> >>>> -Jake >>>> >>>> >>>> On Mon, Mar 17, 2014 at 12:04 PM, Joe Schaefer >>> <joe_schae...@yahoo.com>wrote: >>>> >>>>> Any objections to me making a copy of the site >>>>> tree alongside it called cms-site? I'd like to >>>>> get cracking on knocking up the supporting perl >>>>> for this so we can continue to brainstorm... >>>>> >>>>> >>>>> >>>>> >>>>> > On Sunday, March 16, 2014 1:53 PM, Joe Schaefer >>> <joe_schae...@yahoo.com> >>>>> wrote: >>>>> > > On Sunday, March 16, 2014 1:40 PM, Jake Farrell >>> <jfarr...@apache.org> >>>>> > wrote: >>>>> > >>>>> > Hey Joe >>>>> >> Thanks for reaching out, we chose to go with a site >> generator >>> over the >>>>> >> Apache CMS due to it initially not meeting all of > our >> needs. >>> Our current >>>>> >> needs for the site are >>>>> >> >>>>> >> - Markdown to html conversion >>>>> > >>>>> > >>>>> > That one's easy- its the default for the cms. >>>>> > >>>>> > >>>>> >> - Global variables/config usable throughout in a > template >> >>> based layout >>>>> > >>>>> > >>>>> > Easy too- just create a perl hash somewhere and make it >> available >>> to >>>>> your views. >>>>> > Code your views to pass that hash along to your > templates >> and you >>> are >>>>> all set. >>>>> > Note if your hash contains objects you can make method > calls >> on >>> them in >>>>> your >>>>> > templates. Note: while I wouldn't recommend this in > >> general, >>> for smaller >>>>> > projects like thrift that prefer convenience over > separation >> of >>> concerns >>>>> you can >>>>> > have the django template engine do a pass over your > markdown >>> before >>>>> passing it >>>>> > along to your actual template page, just as is seemingly > >> common to >>> other >>>>> popular >>>>> > site generation software. >>>>> > >>>>> > >>>>> >> - Syntax highlighting >>>>> > >>>>> > >>>>> > Comes with python's markdown support, but some > people >> prefer >>> the look of >>>>> > javascript highlighters. >>>>> > >>>>> >> - Easily include code snippets from our source code > base >>>>> > >>>>> > >>>>> > Statically I hope. No idea how to pull snippets out of > your >> git >>> repo >>>>> via the >>>>> > cms. >>>>> > >>>>> >> - Ability to locally test before committing >>>>> > >>>>> > >>>>> > Belt and suspenders with the cms- you can build your > changes >>> before >>>>> committing >>>>> > with the build scripts, browse your changes before > committing >> via >>> the cms >>>>> > webgui, or simply commit your changes and view the > staging >> site >>> prior to >>>>> live >>>>> > publication (which is a separate step). >>>>> > >>>>> > >>>>> > >>>>> >> >>>>> >> -Jake >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> On Sun, Mar 16, 2014 at 12:44 PM, Joe Schaefer >>>>> > <joe_schae...@yahoo.com>wrote: >>>>> >> >>>>> >>> As it so happens I am interested in working on >> supporting >>>>> >>> thrift's use case as a potential user of > the >> Apache >>> CMS. >>>>> >>> While the CMS works well for massive projects > there >> are >>>>> >>> things it could do better at supporting for > more >> moderate >>>>> >>> sized sites like thrift's. >>>>> >>> >>>>> >>> I'd therefore like to engage you folks in a > >>> brainstorming session >>>>> >>> about what sorts of features you want for your > site >> and >>> to find >>>>> >>> simple ways of supporting those features for > you. >>>>> >>> >>>>> >>> Thanks. >>>>> >> >>>>> >> >>>>> >> >>>>> > >>>>> >>> >> >
