Hi
On 23.03.2014 03:19, Jake Farrell wrote:
Hey Joe
Reviewing the staging site it looks like its almost 99% there other than
some styling within the code snippets. I'm good if we want to cut over to
using this and then work on the additional features that had been mentioned
afterwords. Any objections to cutting over?
I would really like to see this automatic README.md includes from source
tree, to have a real additional feature beside of what we already have
today.
I'm absolutely with you.
Apache projects shall use other Apache projects whenever possible.
On Sat, Mar 22, 2014 at 4:36 PM, Joe Schaefer <joe_schae...@yahoo.com>wrote:
May I suggest a script that creates files in content/
containing a single line
[snippet:path=...]
this sounds like a plan
(with no 'lang' argument) that lets the CMS pull raw content from
thrift.git
and add it directly to the built pages. That way you only need to run the
script that generates these files when a path changes in your thrift.git
repo.
What I had in mind is a automatic search of README.md files across the
source tree and generate the site with these automatically.
Kind of a central controller that maps all the URL's.
no additional work when a new README.md file is added.
however, a script can do this as interim soultion.
You can commit directly to the cms svn tree or via the webgui which does
the same thing.
Great!
What about local preview when doing perl modifications?
-roger
On Saturday, March 22, 2014 4:20 PM, Roger Meier <
ro...@bufferoverflow.ch> wrote:
I still see some issues and still miss to feature mentioned.
Now we have nonoc.ws and I'm able to hack this a bit.
We had no real big issue, the thing I missed was the code snippet thing
and this is in on nanoc.ws and now also on Apache CMS.
The thing we really need is the documentation located within source code
integrated on the web site.
most hosting platforms do render README.md and that's why lot of doc
will stay within the code and we should benefit as well by rendering
that to the web site and enhance the documentation very easily.
I would like to render the 41 README.md files to with the following
pattern, that's a must have before switching to another CMS.
test/README.md => thrift.apache.org/test
test/STATUS.md => thrift.apache.org/test/status
test/keys/README.md => thrift.apache.org/test/keys
lib/<lang>/README.md => thrift.apache.org/lib/<lang>
lib/<lang>/test/README.md => thrift.apache.org/lib/<lang>/test
lib/<lang>/examples/README.md =>
thrift.apache.org/lib/<lang>/examples
compiler/cpp/README.md => thrift.apache.org/compiler
debian/README.md => thrift.apache.org/debian
contrib/<topic>/README.md => thrift.apache.org/<topic>
CPP tutorial seems to be broken on staging.
Do you commit to https://svn.apache.org/repos/asf/thrift/cms-site/trunk
or is it web gui only?
How do you generate the site on your local machine?
Should I reactivate my perl or do we stay with ruby?
Committers and contributors, what do you think?
all the best
-roger
On 22.03.2014 18:07, Joe Schaefer wrote:
Ok I've now done the basic necessities involved in porting
the site over. Links have been adjusted, templates ported, files
created,
snippet feature activated. At this point I'm happy to declare
victory and migrate the thrift live site over, as anything broken that
still remains can be dealt with on a per-page basis with the
bookmarklet.
On Thursday, March 20, 2014 11:19 AM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
Modulo some bizarre content truncation out of git-wip-us.apache.org
the snippet support is now active. At this point I think all the build
support you need in this project to support every type of
brainstorming
activity we've discussed is now there, so it's just up to you
guys to
now familiarize yourselves with the build technology and how to apply
it to your needs. The site builds with snippet support are running
around
10 seconds, which is to be expected with the network interactions
involved.
I'm always willing to answer any questions that come up in the
interim,
but I wish you guys the best of luck in the remaining porting work to
be done.
All I see really are some links that need to have the trailing slashes
removed
and whatever <%= %> template constructs that remain need to be
ported to
django. So it should be smooth sailing, but as I said if you hit any
snags
holler. I'd really like to see the live site tech switched over to
the cms
stuff before Apachecon ends in early April, hint hint.
On Wednesday, March 19, 2014 10:41 PM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> While I don't yet know the url magic to support branches
and tags
in git,
I can say that the snippet support is ready to play with. I
suggest just
focusing on the /index.md page first to see how it works in
practice
because
once it's fully enabled the build will break until all the
snippet
sources
have been altered to conform with the new style.
To turn it on for index.md testing, there are a couple of things
that need
to
happen. First the [snippet:...] marker should be edited to be
[XXXsnippet:...]
to enable testing without breaking the build. Second each marker
contains
a
"token" argument instead of line numbers for more
stability under
edits to the source code. The token argument is an arbitrary
string that
is
used to denote the boundaries of the snippet within the source
code.
ASF::Value::Snippet will pull out the lines from the source file
between
the
lines that contain the token argument and format them as a code
snippet
embedded
into markdown. To get started I'll copy over the code
highlighting css
from
www.apache.org but note that this is all based on python pygments
and there
are
a variety of online css files to choose from if this one is not
to the
group's liking.
On Wednesday, March 19, 2014 4:16 PM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> Sorry I need to clarify with an example: look at
http://thrift.staging.apache.org/tutorial/ versus
http://thrift.staging.apache.org/tutorial.html. The
former is generated from content/tutorial/index.html
and the latter from content/tutorial.md. The latter file
needs to be removed from the tree now that it's content
has been ported to content/tutorial/index.html. This
uses django template inheritance to maintain simplicity.
Other directories that follow a similar pattern should
also be ported in the same fashion.
On Wednesday, March 19, 2014 9:46 AM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> If someone's looking for a way to help with
the porting
effort, consider tackling the creation of an index.html
page in each subdir of the content/ tree (other than
the
base which has an /index.md). The contents of the file
should be just
{% include 'default' %}
(see the docs dir for an example page that can be
copied)
and the build system will take care of generating the
page
in a consistent fashion with the rest of the site.
Once
that is done there are a lot of pages that function
like
index pages but are written in markdown that can be
deleted,
like docs.md and its ilk. Nanoc transforms that file
into
/docs/index.html but that's kinda crufty and not
supported
by the CMS, which only lets you alter file extensions
not paths.
On Wednesday, March 19, 2014 8:54 AM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> I just pencilled in snippet preprocessing
support into
the thrift builds. Once Jake and I have settled
on the
feature set of ASF::Value::Snippet, and we have
ported
the snippet calling text within the markdown pages
to the new format, things will just work (knock on
wood).
On Wednesday, March 19, 2014 6:48 AM, Joseph
Schaefer
<joe_schae...@yahoo.com> wrote:
> Sure I understand that need. If
there's a way
of
pulling the
document
sources directly from the web I can parse the
content
to
look for
snippet
markers and make those available to the
template
system.
That's
pretty
much
how www.apache.org is generated.
Sent from my iPhone
On Mar 19, 2014, at 5:28 AM, Henrique
Mendonça
<henri...@apache.org>
wrote:
Hi Joe,
Thanks for your effort. I just wanted to
add that
the
code
snippets
are a
crucial feature for this project, which
has way
more
target
languages
than
active contributors. In our case,
reducing manual
maintenance
costs is
much
more important than the total site build
time.
Perhaps
we
can
also
solve
this in the CMS, though.
- Henrique
On 19 March 2014 03:17, Joe Schaefer
<joe_schae...@yahoo.com>
wrote:
A couple more thoughts regarding the
background on
the
CMS...
the bulk of the focus of the CMS up
until now
has
been
to
support really big sites like maven
and
openoffice,
both
of
which have websites well over 5GB
worth of
text
files.
The
build system was hollowed out and
revamped to
support
external
builds in other programming
languages, as well
as
scaling up
the perl builds to provide about 8
child
processes
that
build
the site in parallel by default.
Full site
build
times
for
openoffice went from over 30 minutes
to
between 5
to 10
minutes
depending on disk activity, which
made
experimentation
with
site-wide
changes feasible on a reasonable
timeframe.
At
first
the
project
was
in
the habit of running the builds
locally before
checking
in
their
changes
because the delay in feedback
post-commit was
unreasonable,
but
with
smart
use of SSI the were able to cut down
on the
number
of
changes
that
required
full site builds and hence no longer
bother
with
the
whole
pre-build
mumbo
jumbo before checking in changes.
The CMS is
designed
to
only
build
pages
that were altered, and if you just
edit a page
or
two
only
those
pages
and
the pages that depend on them will
be built.
Now that I've turned my
attention to
thrift,
there
are
several
aspects
that are better off being retuned
based on the
modest
size of
the
site.
First of all you have a sitemap url
that lists
all
of
your
markdown-based
pages, which would be a nonstarter
for a site
like
openoffice.
The
implications of the sitemap url are
that every
page
needs to
be
built
in
memory just to generate that page,
which leads
to a
lot
of
redundancy
in a
build system designed to run as
forked child
processes.
We
can do
better...
I've done two things to make
this work
well-
first I
made
the
number of
child processes "aka
runners"
tunable on
a
per-project
basis,
and since the
sitemap is going to build every page
anyway as
the
slowest
page to
build, I
memoized the main view that builds
the
markdown
pages
for
thrift.
The
best
results are when there's only
one runner
so we
get
the
full
benefit
of
memoization: typical site build
times are
consistently
under
2
seconds
now.
You'll appreciate why I am such
a
performance
stickler
for the
CMS
when
you start using the bookmarklet,
since builds
are
the
only
asynchronous
part of the process from making a
change to a
page
and
publishing
the
results. Ideally when the system is
working
well
the
builds
happen
before
you have time to go looking for
them, but you
do
need to
be
mindful of
not
publishing before the built pages
have been
committed
back to
svn
by
buildbot.
On Monday, March 17, 2014 11:59
PM, Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
Ok there's enough of an
idea about
how
to
port
the
rest of
the site's
pages over based on the porting
work
I've
already
done
that
it's time
for me to step back and let you
guys catch
up.
Basically
the
idea
is
that while
there is templating support in
the
markdown
sources,
it's
fairly limited
compared to what nanoc offers,
but instead
of
writing a
lot of
complex
template
code into your sources, with the
cms you
just
add
code to
lib/view.pmand/or
lib/path.pm to keep the document
sources
simple
and
just
add
more
template
variables. What I just finished
in
lib/view.pm
and
lib/path.pm is
directory
listing support for /docs.html
but it can
be
carried
over
to
similar
pages.
On Monday, March 17, 2014
9:16 PM, Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
Interesting ideas,
thanks. FWIW I
think
that
sort
of thing might be best
supported as a
periodic
cron
if we can write something
stable
enough,
because
pulling
stuff out of the git repo is
something
the
CMS
isn't
going
to do all that well being an
svn
application.
The CMS build results are
available at
http://thrift.staging.apache.org/
feel free to play with the
sources in
repos/asf/thrift/cms-site/trunk.
It's very fast, probably
an order
of
magnitude
faster
than
your current
tech.
Whole site builds in two
seconds;
fractions
of a
second
for
typical
mods to
content/ markdown pages.
The CMS bookmarklet is
compatible with
the
staging
site
FWIW.
On Monday, March 17,
2014 7:40
PM,
Roger
Meier
<ro...@bufferoverflow.ch> wrote:
Hi Joe & Co
thanks for this test
run with
Apache
CMS!
What I would love to
see is this:
Source file: => Web
Site URL:
test/README.md =>
thrift.apache.org/test
test/STATUS.md =>
thrift.apache.org/test/status
test/keys/README.md
=>
thrift.apache.org/test/keys
lib/<lang>/README.md =>
thrift.apache.org/lib/<lang>
lib/<lang>/test/README.md
=>
thrift.apache.org/lib/<lang>/test
lib/<lang>/examples/README.md
=>
thrift.apache.org/lib/<lang>/examples
compiler/cpp/README.md
=>
thrift.apache.org/compiler
debian/README.md =>
thrift.apache.org/debian
contrib/<topic>/README.md
=>
thrift.apache.org/<topic>
just made the following
patch to
rename all
file
to
.md
within our
source
tree:
https://issues.apache.org/jira/browse/THRIFT-2407
... we have *41*
md's within
the
source
tree
and
that's the
place
where
people look for
documentation
first.
=>
simple to
manage
Would be user friendly
to have
all of
this
online
with
each release
with the URL layout
mentioned
above.
just received the
buildboot...
Do you have kind of a
preview of
your
prototype?
all the best!
-roger
;-r
Quoting Jake Farrell
<jfarr...@apache.org>:
No objections from
me
-Jake
On Mon, Mar 17,
2014 at
12:04 PM,
Joe
Schaefer
<joe_schae...@yahoo.com>wrote:
Any objections
to me
making a
copy
of
the
site
tree alongside
it called
cms-site?
I'd
like
to
get cracking
on knocking
up
the
supporting
perl
for this so we
can
continue
to
brainstorm...
On Sunday,
March 16,
2014
1:53
PM,
Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
On
Sunday, March
16,
2014
1:40
PM,
Jake
Farrell
<jfarr...@apache.org>
wrote:
Hey Joe
Thanks
for
reaching
out, we
chose
to
go
with a site
generator
over the
Apache
CMS due to
it
initially
not
meeting
all of
our
needs.
Our current
needs
for the site
are
-
Markdown to html
conversion
That
one's easy-
its
the
default
for
the
cms.
- Global
variables/config
usable
throughout
in a
template
based layout
Easy too-
just create
a
perl
hash
somewhere and
make it
available
to
your views.
Code your
views to
pass
that
hash
along to
your
templates
and you
are
all set.
Note if your
hash
contains
objects
you can
make
method
calls
on
them in
your
templates.
Note:
while I
wouldn't
recommend this in
general,
for smaller
projects
like thrift
that
prefer
convenience
over
separation
of
concerns
you can
have the
django
template
engine
do a
pass
over
your
markdown
before
passing it
along to
your actual
template
page,
just
as is
seemingly
common to
other
popular
site
generation
software.
- Syntax
highlighting
Comes with
python's
markdown
support,
but
some
people
prefer
the look of
javascript
highlighters.
- Easily
include
code
snippets
from
our
source code
base
Statically I
hope. No
idea
how
to
pull
snippets out of
your
git
repo
via the
cms.
-
Ability to
locally
test
before
committing
Belt and
suspenders
with
the
cms- you
can
build
your
changes
before
committing
with the
build
scripts,
browse
your
changes
before
committing
via
the cms
webgui, or
simply
commit
your
changes
and
view
the
staging
site
prior to
live
publication
(which is
a
separate
step).
-Jake
On Sun,
Mar 16,
2014 at
12:44 PM,
Joe
Schaefer
<joe_schae...@yahoo.com>wrote:
As
it so
happens I
am
interested
in
working on
supporting
thrift's
use
case as
a
potential
user of
the
Apache
CMS.
While the CMS
works
well
for
massive
projects
there
are
things it
could do
better at
supporting
for
more
moderate
sized sites
like
thrift's.
I'd
therefore
like
to
engage
you
folks in a
brainstorming session
about what
sorts of
features
you
want
for your
site
and
to find
simple ways of
supporting
those
features for
you.
Thanks.
