Sorry I need to clarify with an example: look at
http://thrift.staging.apache.org/tutorial/ versus
http://thrift.staging.apache.org/tutorial.html. The
former is generated from content/tutorial/index.html
and the latter from content/tutorial.md. The latter file
needs to be removed from the tree now that it's content
has been ported to content/tutorial/index.html. This
uses django template inheritance to maintain simplicity.
Other directories that follow a similar pattern should
also be ported in the same fashion.
> On Wednesday, March 19, 2014 9:46 AM, Joe Schaefer <joe_schae...@yahoo.com>
> wrote:
> > If someone's looking for a way to help with the porting
> effort, consider tackling the creation of an index.html
> page in each subdir of the content/ tree (other than the
> base which has an /index.md). The contents of the file
> should be just
>
> {% include 'default' %}
>
> (see the docs dir for an example page that can be copied)
> and the build system will take care of generating the page
> in a consistent fashion with the rest of the site. Once
> that is done there are a lot of pages that function like
> index pages but are written in markdown that can be deleted,
> like docs.md and its ilk. Nanoc transforms that file into
> /docs/index.html but that's kinda crufty and not supported
> by the CMS, which only lets you alter file extensions not paths.
>
>
>
>
>> On Wednesday, March 19, 2014 8:54 AM, Joe Schaefer
> <joe_schae...@yahoo.com> wrote:
>> > I just pencilled in snippet preprocessing support into
>> the thrift builds. Once Jake and I have settled on the
>> feature set of ASF::Value::Snippet, and we have ported
>> the snippet calling text within the markdown pages
>> to the new format, things will just work (knock on wood).
>>
>>
>>
>>
>>> On Wednesday, March 19, 2014 6:48 AM, Joseph Schaefer
>> <joe_schae...@yahoo.com> wrote:
>>> > Sure I understand that need. If there's a way of pulling the
>
>> document
>>> sources directly from the web I can parse the content to look for
> snippet
>>> markers and make those available to the template system. That's
> pretty
>> much
>>> how www.apache.org is generated.
>>>
>>> Sent from my iPhone
>>>
>>>
>>>> On Mar 19, 2014, at 5:28 AM, Henrique Mendonça
>> <henri...@apache.org>
>>> wrote:
>>>>
>>>> Hi Joe,
>>>>
>>>> Thanks for your effort. I just wanted to add that the code
> snippets
>> are a
>>>> crucial feature for this project, which has way more target
> languages
>> than
>>>> active contributors. In our case, reducing manual maintenance
> costs is
>> much
>>>> more important than the total site build time. Perhaps we can
> also
>> solve
>>>> this in the CMS, though.
>>>>
>>>> - Henrique
>>>>
>>>>
>>>>> On 19 March 2014 03:17, Joe Schaefer
>> <joe_schae...@yahoo.com>
>>> wrote:
>>>>>
>>>>> A couple more thoughts regarding the background on the CMS...
>>>>> the bulk of the focus of the CMS up until now has been to
>>>>> support really big sites like maven and openoffice, both of
>>>>> which have websites well over 5GB worth of text files. The
>>>>> build system was hollowed out and revamped to support
> external
>>>>> builds in other programming languages, as well as scaling up
>>>>> the perl builds to provide about 8 child processes that build
>>>>> the site in parallel by default. Full site build times for
>>>>> openoffice went from over 30 minutes to between 5 to 10
> minutes
>>>>> depending on disk activity, which made experimentation with
>> site-wide
>>>>> changes feasible on a reasonable timeframe. At first the
> project
>> was
>>> in
>>>>> the habit of running the builds locally before checking in
> their
>>> changes
>>>>> because the delay in feedback post-commit was unreasonable,
> but
>> with
>>> smart
>>>>> use of SSI the were able to cut down on the number of changes
> that
>>
>>> required
>>>>> full site builds and hence no longer bother with the whole
>> pre-build
>>> mumbo
>>>>> jumbo before checking in changes. The CMS is designed to
> only
>> build
>>> pages
>>>>> that were altered, and if you just edit a page or two only
> those
>> pages
>>> and
>>>>> the pages that depend on them will be built.
>>>>>
>>>>> Now that I've turned my attention to thrift, there are
> several
>>
>>> aspects
>>>>> that are better off being retuned based on the modest size of
> the
>> site.
>>>>> First of all you have a sitemap url that lists all of your
>>> markdown-based
>>>>> pages, which would be a nonstarter for a site like
> openoffice.
>> The
>>>>> implications of the sitemap url are that every page needs to
> be
>> built
>>> in
>>>>> memory just to generate that page, which leads to a lot of
>> redundancy
>>> in a
>>>>> build system designed to run as forked child processes. We
> can do
>>
>>> better...
>>>>>
>>>>> I've done two things to make this work well- first I made
> the
>>> number of
>>>>> child processes "aka runners" tunable on a
> per-project
>> basis,
>>> and since the
>>>>> sitemap is going to build every page anyway as the slowest
> page to
>>
>>> build, I
>>>>> memoized the main view that builds the markdown pages for
> thrift.
>> The
>>> best
>>>>> results are when there's only one runner so we get the
> full
>> benefit
>>> of
>>>>> memoization: typical site build times are consistently under
> 2
>> seconds
>>> now.
>>>>>
>>>>> You'll appreciate why I am such a performance stickler
> for the
>> CMS
>>> when
>>>>> you start using the bookmarklet, since builds are the only
>> asynchronous
>>>>> part of the process from making a change to a page and
> publishing
>> the
>>>>> results. Ideally when the system is working well the builds
>> happen
>>> before
>>>>> you have time to go looking for them, but you do need to be
>> mindful of
>>> not
>>>>> publishing before the built pages have been committed back to
> svn
>> by
>>>>> buildbot.
>>>>>
>>>>>
>>>>>
>>>>>> On Monday, March 17, 2014 11:59 PM, Joe Schaefer
>>> <joe_schae...@yahoo.com>
>>>>> wrote:
>>>>>>> Ok there's enough of an idea about how to port
> the
>> rest of
>>> the site's
>>>>>> pages over based on the porting work I've already
> done
>> that
>>> it's time
>>>>>> for me to step back and let you guys catch up. Basically
> the
>> idea
>>> is
>>>>> that while
>>>>>> there is templating support in the markdown sources,
> it's
>>> fairly limited
>>>>>> compared to what nanoc offers, but instead of writing a
> lot of
>>
>>> complex
>>>>> template
>>>>>> code into your sources, with the cms you just add code to
>
>>> lib/view.pmand/or
>>>>>> lib/path.pm to keep the document sources simple and just
> add
>> more
>>>>> template
>>>>>> variables. What I just finished in lib/view.pm and
>> lib/path.pm is
>>>>> directory
>>>>>> listing support for /docs.html but it can be carried over
> to
>>> similar
>>>>> pages.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Monday, March 17, 2014 9:16 PM, Joe Schaefer
>>>>>> <joe_schae...@yahoo.com> wrote:
>>>>>>>> Interesting ideas, thanks. FWIW I think that
> sort
>>>>>>> of thing might be best supported as a periodic cron
>>>>>>> if we can write something stable enough, because
> pulling
>>>>>>> stuff out of the git repo is something the CMS
> isn't
>> going
>>>>>>> to do all that well being an svn application.
>>>>>>>
>>>>>>> The CMS build results are available at
>>>>> http://thrift.staging.apache.org/
>>>>>>> feel free to play with the sources in
>>> repos/asf/thrift/cms-site/trunk.
>>>>>>> It's very fast, probably an order of magnitude
> faster
>> than
>>> your current
>>>>>>
>>>>>>> tech.
>>>>>>> Whole site builds in two seconds; fractions of a
> second
>> for
>>> typical
>>>>> mods to
>>>>>>
>>>>>>> content/ markdown pages.
>>>>>>>
>>>>>>> The CMS bookmarklet is compatible with the staging
> site
>> FWIW.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On Monday, March 17, 2014 7:40 PM, Roger Meier
>>>>>>> <ro...@bufferoverflow.ch> wrote:
>>>>>>>>> Hi Joe & Co
>>>>>>>>
>>>>>>>> thanks for this test run with Apache CMS!
>>>>>>>>
>>>>>>>> What I would love to see is this:
>>>>>>>>
>>>>>>>> Source file: => Web Site URL:
>>>>>>>> test/README.md => thrift.apache.org/test
>>>>>>>> test/STATUS.md =>
> thrift.apache.org/test/status
>>>>>>>> test/keys/README.md =>
> thrift.apache.org/test/keys
>>>>>>>> lib/<lang>/README.md =>
>>> thrift.apache.org/lib/<lang>
>>>>>>>> lib/<lang>/test/README.md =>
>>>>>>> thrift.apache.org/lib/<lang>/test
>>>>>>>> lib/<lang>/examples/README.md =>
>>>>>>>> thrift.apache.org/lib/<lang>/examples
>>>>>>>> compiler/cpp/README.md =>
>> thrift.apache.org/compiler
>>>>>>>> debian/README.md => thrift.apache.org/debian
>>>>>>>> contrib/<topic>/README.md =>
>>> thrift.apache.org/<topic>
>>>>>>>>
>>>>>>>> just made the following patch to rename all file
> to
>> .md
>>> within our
>>>>>> source
>>>>>>>> tree:
>> https://issues.apache.org/jira/browse/THRIFT-2407
>>>>>>>> ... we have *41* md's within the source tree
> and
>>> that's the
>>>>>> place
>>>>>>> where
>>>>>>>> people look for documentation first. =>
> simple to
>>> manage
>>>>>>>> Would be user friendly to have all of this
> online
>> with
>>> each release
>>>>>>>> with the URL layout mentioned above.
>>>>>>>>
>>>>>>>> just received the buildboot...
>>>>>>>> Do you have kind of a preview of your prototype?
>>>>>>>>
>>>>>>>> all the best!
>>>>>>>> -roger
>>>>>>>> ;-r
>>>>>>>>
>>>>>>>>
>>>>>>>> Quoting Jake Farrell
> <jfarr...@apache.org>:
>>>>>>>>
>>>>>>>>> No objections from me
>>>>>>>>>
>>>>>>>>> -Jake
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mon, Mar 17, 2014 at 12:04 PM, Joe
> Schaefer
>>>>>>>> <joe_schae...@yahoo.com>wrote:
>>>>>>>>>
>>>>>>>>>> Any objections to me making a copy of
> the
>> site
>>>>>>>>>> tree alongside it called cms-site?
> I'd
>> like
>>> to
>>>>>>>>>> get cracking on knocking up the
> supporting
>> perl
>>>>>>>>>> for this so we can continue to
> brainstorm...
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Sunday, March 16, 2014 1:53 PM,
> Joe
>> Schaefer
>>>>>>>> <joe_schae...@yahoo.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>> On Sunday, March 16, 2014 1:40
> PM,
>> Jake
>>> Farrell
>>>>>>>> <jfarr...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hey Joe
>>>>>>>>>>>> Thanks for reaching out, we chose
> to
>> go
>>> with a site
>>>>>>> generator
>>>>>>>> over the
>>>>>>>>>>>> Apache CMS due to it initially
> not
>> meeting
>>> all of
>>>>>> our
>>>>>>> needs.
>>>>>>>> Our current
>>>>>>>>>>>> needs for the site are
>>>>>>>>>>>>
>>>>>>>>>>>> - Markdown to html conversion
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> That one's easy- its the default
> for
>> the
>>> cms.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> - Global variables/config usable
>> throughout
>>> in a
>>>>>> template
>>>>>>>
>>>>>>>> based layout
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Easy too- just create a perl hash
>> somewhere and
>>> make it
>>>>>>> available
>>>>>>>> to
>>>>>>>>>> your views.
>>>>>>>>>>> Code your views to pass that hash
> along to
>> your
>>>>>> templates
>>>>>>> and you
>>>>>>>> are
>>>>>>>>>> all set.
>>>>>>>>>>> Note if your hash contains objects
> you can
>> make
>>> method
>>>>>> calls
>>>>>>> on
>>>>>>>> them in
>>>>>>>>>> your
>>>>>>>>>>> templates. Note: while I
> wouldn't
>>> recommend this in
>>>>>>
>>>>>>> general,
>>>>>>>> for smaller
>>>>>>>>>>> projects like thrift that prefer
>> convenience
>>> over
>>>>>> separation
>>>>>>> of
>>>>>>>> concerns
>>>>>>>>>> you can
>>>>>>>>>>> have the django template engine do a
> pass
>> over
>>> your
>>>>>> markdown
>>>>>>>> before
>>>>>>>>>> passing it
>>>>>>>>>>> along to your actual template page,
> just
>> as is
>>> seemingly
>>>>>>
>>>>>>> common to
>>>>>>>> other
>>>>>>>>>> popular
>>>>>>>>>>> site generation software.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> - Syntax highlighting
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Comes with python's markdown
> support,
>> but
>>> some
>>>>>> people
>>>>>>> prefer
>>>>>>>> the look of
>>>>>>>>>>> javascript highlighters.
>>>>>>>>>>>
>>>>>>>>>>>> - Easily include code snippets
> from
>> our
>>> source code
>>>>>> base
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Statically I hope. No idea how to
> pull
>>> snippets out of
>>>>>> your
>>>>>>> git
>>>>>>>> repo
>>>>>>>>>> via the
>>>>>>>>>>> cms.
>>>>>>>>>>>
>>>>>>>>>>>> - Ability to locally test before
>> committing
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Belt and suspenders with the cms- you
> can
>> build
>>> your
>>>>>> changes
>>>>>>>> before
>>>>>>>>>> committing
>>>>>>>>>>> with the build scripts, browse your
>> changes
>>> before
>>>>>> committing
>>>>>>> via
>>>>>>>> the cms
>>>>>>>>>>> webgui, or simply commit your changes
> and
>> view
>>> the
>>>>>> staging
>>>>>>> site
>>>>>>>> prior to
>>>>>>>>>> live
>>>>>>>>>>> publication (which is a separate
> step).
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> -Jake
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Mar 16, 2014 at 12:44 PM,
> Joe
>>> Schaefer
>>>>>>>>>>> <joe_schae...@yahoo.com>wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> As it so happens I am
> interested
>> in
>>> working on
>>>>>>> supporting
>>>>>>>>>>>>> thrift's use case as a
>> potential
>>> user of
>>>>>> the
>>>>>>> Apache
>>>>>>>> CMS.
>>>>>>>>>>>>> While the CMS works well for
>> massive
>>> projects
>>>>>> there
>>>>>>> are
>>>>>>>>>>>>> things it could do better at
>> supporting
>>> for
>>>>>> more
>>>>>>> moderate
>>>>>>>>>>>>> sized sites like
> thrift's.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I'd therefore like to
> engage
>> you
>>> folks in a
>>>>>>
>>>>>>>> brainstorming session
>>>>>>>>>>>>> about what sorts of features
> you
>> want
>>> for your
>>>>>> site
>>>>>>> and
>>>>>>>> to find
>>>>>>>>>>>>> simple ways of supporting
> those
>>> features for
>>>>>> you.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks.
>>>>>
>>>
>>
>
