On Sunday, March 23, 2014 9:15 AM, Roger Meier <ro...@bufferoverflow.ch> wrote:
Hi
On 23.03.2014 03:19, Jake Farrell wrote:
Hey Joe
Reviewing the staging site it looks like its almost 99% there other than
some styling within the code snippets. I'm good if we want to cut over
to
using this and then work on the additional features that had been mentioned
afterwords. Any objections to cutting over?
I would really like to see this automatic README.md includes from source
tree, to have a real additional feature beside of what we already have
today.
I'm absolutely with you.
Apache projects shall use other Apache projects whenever possible.
On Sat, Mar 22, 2014 at 4:36 PM, Joe Schaefer
<joe_schae...@yahoo.com>wrote:
May I suggest a script that creates files in content/
containing a single line
[snippet:path=...]
this sounds like a plan
(with no 'lang' argument) that lets the CMS pull raw content
from
thrift.git
and add it directly to the built pages. That way you only need to run
the
script that generates these files when a path changes in your
thrift.git
repo.
What I had in mind is a automatic search of README.md files across the
source tree and generate the site with these automatically.
Kind of a central controller that maps all the URL's.
no additional work when a new README.md file is added.
however, a script can do this as interim soultion.
You can commit directly to the cms svn tree or via the webgui which
does
the same thing.
Great!
What about local preview when doing perl modifications?
-roger
On Saturday, March 22, 2014 4:20 PM, Roger Meier <
ro...@bufferoverflow.ch> wrote:
I still see some issues and still miss to feature mentioned.
Now we have nonoc.ws and I'm able to hack this a bit.
We had no real big issue, the thing I missed was the code snippet
thing
and this is in on nanoc.ws and now also on Apache CMS.
The thing we really need is the documentation located within source
code
integrated on the web site.
most hosting platforms do render README.md and that's why lot
of doc
will stay within the code and we should benefit as well by
rendering
that to the web site and enhance the documentation very easily.
I would like to render the 41 README.md files to with the following
pattern, that's a must have before switching to another CMS.
test/README.md => thrift.apache.org/test
test/STATUS.md => thrift.apache.org/test/status
test/keys/README.md => thrift.apache.org/test/keys
lib/<lang>/README.md => thrift.apache.org/lib/<lang>
lib/<lang>/test/README.md =>
thrift.apache.org/lib/<lang>/test
lib/<lang>/examples/README.md =>
thrift.apache.org/lib/<lang>/examples
compiler/cpp/README.md => thrift.apache.org/compiler
debian/README.md => thrift.apache.org/debian
contrib/<topic>/README.md =>
thrift.apache.org/<topic>
CPP tutorial seems to be broken on staging.
Do you commit to
https://svn.apache.org/repos/asf/thrift/cms-site/trunk
or is it web gui only?
How do you generate the site on your local machine?
Should I reactivate my perl or do we stay with ruby?
Committers and contributors, what do you think?
all the best
-roger
On 22.03.2014 18:07, Joe Schaefer wrote:
Ok I've now done the basic necessities involved in
porting
the site over. Links have been adjusted, templates ported,
files
created,
snippet feature activated. At this point I'm happy to
declare
victory and migrate the thrift live site over, as anything
broken that
still remains can be dealt with on a per-page basis with the
bookmarklet.
On Thursday, March 20, 2014 11:19 AM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
Modulo some bizarre content truncation out of
git-wip-us.apache.org
the snippet support is now active. At this point I think
all the build
support you need in this project to support every type of
brainstorming
activity we've discussed is now there, so it's
just up to you
guys to
now familiarize yourselves with the build technology and
how to apply
it to your needs. The site builds with snippet support
are running
around
10 seconds, which is to be expected with the network
interactions
involved.
I'm always willing to answer any questions that come
up in the
interim,
but I wish you guys the best of luck in the remaining
porting work to
be done.
All I see really are some links that need to have the
trailing slashes
removed
and whatever <%= %> template constructs that remain
need to be
ported to
django. So it should be smooth sailing, but as I said if
you hit any
snags
holler. I'd really like to see the live site tech
switched over to
the cms
stuff before Apachecon ends in early April, hint hint.
On Wednesday, March 19, 2014 10:41 PM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> While I don't yet know the url magic to
support branches
and tags
in git,
I can say that the snippet support is ready to play
with. I
suggest just
focusing on the /index.md page first to see how it
works in
practice
because
once it's fully enabled the build will break
until all the
snippet
sources
have been altered to conform with the new style.
To turn it on for index.md testing, there are a
couple of things
that need
to
happen. First the [snippet:...] marker should be
edited to be
[XXXsnippet:...]
to enable testing without breaking the build.
Second each marker
contains
a
"token" argument instead of line numbers
for more
stability under
edits to the source code. The token argument is an
arbitrary
string that
is
used to denote the boundaries of the snippet within
the source
code.
ASF::Value::Snippet will pull out the lines from
the source file
between
the
lines that contain the token argument and format
them as a code
snippet
embedded
into markdown. To get started I'll copy over
the code
highlighting css
from
www.apache.org but note that this is all based on
python pygments
and there
are
a variety of online css files to choose from if
this one is not
to the
group's liking.
On Wednesday, March 19, 2014 4:16 PM, Joe
Schaefer
<joe_schae...@yahoo.com> wrote:
> Sorry I need to clarify with an example:
look at
http://thrift.staging.apache.org/tutorial/
versus
http://thrift.staging.apache.org/tutorial.html. The
former is generated from
content/tutorial/index.html
and the latter from content/tutorial.md. The
latter file
needs to be removed from the tree now that
it's content
has been ported to
content/tutorial/index.html. This
uses django template inheritance to maintain
simplicity.
Other directories that follow a similar
pattern should
also be ported in the same fashion.
On Wednesday, March 19, 2014 9:46 AM, Joe
Schaefer
<joe_schae...@yahoo.com> wrote:
> If someone's looking for a way
to help with
the porting
effort, consider tackling the creation of
an index.html
page in each subdir of the content/ tree
(other than
the
base which has an /index.md). The
contents of the file
should be just
{% include 'default' %}
(see the docs dir for an example page
that can be
copied)
and the build system will take care of
generating the
page
in a consistent fashion with the rest of
the site.
Once
that is done there are a lot of pages
that function
like
index pages but are written in markdown
that can be
deleted,
like docs.md and its ilk. Nanoc
transforms that file
into
/docs/index.html but that's kinda
crufty and not
supported
by the CMS, which only lets you alter
file extensions
not paths.
On Wednesday, March 19, 2014 8:54
AM, Joe Schaefer
<joe_schae...@yahoo.com> wrote:
> I just pencilled in snippet
preprocessing
support into
the thrift builds. Once Jake and I
have settled
on the
feature set of ASF::Value::Snippet,
and we have
ported
the snippet calling text within the
markdown pages
to the new format, things will just
work (knock on
wood).
On Wednesday, March 19, 2014
6:48 AM, Joseph
Schaefer
<joe_schae...@yahoo.com>
wrote:
> Sure I understand that
need. If
there's a way
of
pulling the
document
sources directly from the web I
can parse the
content
to
look for
snippet
markers and make those
available to the
template
system.
That's
pretty
much
how www.apache.org is
generated.
Sent from my iPhone
On Mar 19, 2014, at 5:28
AM, Henrique
Mendonça
<henri...@apache.org>
wrote:
Hi Joe,
Thanks for your effort. I
just wanted to
add that
the
code
snippets
are a
crucial feature for this
project, which
has way
more
target
languages
than
active contributors. In
our case,
reducing manual
maintenance
costs is
much
more important than the
total site build
time.
Perhaps
we
can
also
solve
this in the CMS, though.
- Henrique
On 19 March 2014
03:17, Joe Schaefer
<joe_schae...@yahoo.com>
wrote:
A couple more thoughts
regarding the
background on
the
CMS...
the bulk of the focus
of the CMS up
until now
has
been
to
support really big
sites like maven
and
openoffice,
both
of
which have websites
well over 5GB
worth of
text
files.
The
build system was
hollowed out and
revamped to
support
external
builds in other
programming
languages, as well
as
scaling up
the perl builds to
provide about 8
child
processes
that
build
the site in parallel
by default.
Full site
build
times
for
openoffice went from
over 30 minutes
to
between 5
to 10
minutes
depending on disk
activity, which
made
experimentation
with
site-wide
changes feasible on a
reasonable
timeframe.
At
first
the
project
was
in
the habit of running
the builds
locally before
checking
in
their
changes
because the delay in
feedback
post-commit was
unreasonable,
but
with
smart
use of SSI the were
able to cut down
on the
number
of
changes
that
required
full site builds and
hence no longer
bother
with
the
whole
pre-build
mumbo
jumbo before checking
in changes.
The CMS is
designed
to
only
build
pages
that were altered, and
if you just
edit a page
or
two
only
those
pages
and
the pages that depend
on them will
be built.
Now that I've
turned my
attention to
thrift,
there
are
several
aspects
that are better off
being retuned
based on the
modest
size of
the
site.
First of all you have
a sitemap url
that lists
all
of
your
markdown-based
pages, which would be
a nonstarter
for a site
like
openoffice.
The
implications of the
sitemap url are
that every
page
needs to
be
built
in
memory just to
generate that page,
which leads
to a
lot
of
redundancy
in a
build system designed
to run as
forked child
processes.
We
can do
better...
I've done two
things to make
this work
well-
first I
made
the
number of
child processes
"aka
runners"
tunable on
a
per-project
basis,
and since the
sitemap is going to
build every page
anyway as
the
slowest
page to
build, I
memoized the main view
that builds
the
markdown
pages
for
thrift.
The
best
results are when
there's only
one runner
so we
get
the
full
benefit
of
memoization: typical
site build
times are
consistently
under
2
seconds
now.
You'll appreciate
why I am such
a
performance
stickler
for the
CMS
when
you start using the
bookmarklet,
since builds
are
the
only
asynchronous
part of the process
from making a
change to a
page
and
publishing
the
results. Ideally when
the system is
working
well
the
builds
happen
before
you have time to go
looking for
them, but you
do
need to
be
mindful of
not
publishing before the
built pages
have been
committed
back to
svn
by
buildbot.
On Monday, March
17, 2014 11:59
PM, Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
Ok there's
enough of an
idea about
how
to
port
the
rest of
the site's
pages over based
on the porting
work
I've
already
done
that
it's time
for me to step
back and let you
guys catch
up.
Basically
the
idea
is
that while
there is
templating support in
the
markdown
sources,
it's
fairly limited
compared to what
nanoc offers,
but instead
of
writing a
lot of
complex
template
code into your
sources, with the
cms you
just
add
code to
lib/view.pmand/or
lib/path.pm to
keep the document
sources
simple
and
just
add
more
template
variables. What I
just finished
in
lib/view.pm
and
lib/path.pm is
directory
listing support
for /docs.html
but it can
be
carried
over
to
similar
pages.
On Monday,
March 17, 2014
9:16 PM, Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
Interesting ideas,
thanks. FWIW I
think
that
sort
of thing might
be best
supported as a
periodic
cron
if we can
write something
stable
enough,
because
pulling
stuff out of
the git repo is
something
the
CMS
isn't
going
to do all that
well being an
svn
application.
The CMS build
results are
available at
http://thrift.staging.apache.org/
feel free to
play with the
sources in
repos/asf/thrift/cms-site/trunk.
It's very
fast, probably
an order
of
magnitude
faster
than
your current
tech.
Whole site
builds in two
seconds;
fractions
of a
second
for
typical
mods to
content/
markdown pages.
The CMS
bookmarklet is
compatible with
the
staging
site
FWIW.
On
Monday, March 17,
2014 7:40
PM,
Roger
Meier
<ro...@bufferoverflow.ch> wrote:
Hi Joe
& Co
thanks
for this test
run with
Apache
CMS!
What I
would love to
see is this:
Source
file: => Web
Site URL:
test/README.md =>
thrift.apache.org/test
test/STATUS.md =>
thrift.apache.org/test/status
test/keys/README.md
=>
thrift.apache.org/test/keys
lib/<lang>/README.md =>
thrift.apache.org/lib/<lang>
lib/<lang>/test/README.md
=>
thrift.apache.org/lib/<lang>/test
lib/<lang>/examples/README.md
=>
thrift.apache.org/lib/<lang>/examples
compiler/cpp/README.md
=>
thrift.apache.org/compiler
debian/README.md =>
thrift.apache.org/debian
contrib/<topic>/README.md
=>
thrift.apache.org/<topic>
just made
the following
patch to
rename all
file
to
.md
within our
source
tree:
https://issues.apache.org/jira/browse/THRIFT-2407
... we
have *41*
md's within
the
source
tree
and
that's the
place
where
people
look for
documentation
first.
=>
simple to
manage
Would be
user friendly
to have
all of
this
online
with
each release
with the
URL layout
mentioned
above.
just
received the
buildboot...
Do you
have kind of a
preview of
your
prototype?
all the
best!
-roger
;-r
Quoting
Jake Farrell
<jfarr...@apache.org>:
No
objections from
me
-Jake
On
Mon, Mar 17,
2014 at
12:04 PM,
Joe
Schaefer
<joe_schae...@yahoo.com>wrote:
Any objections
to me
making a
copy
of
the
site
tree alongside
it called
cms-site?
I'd
like
to
get cracking
on knocking
up
the
supporting
perl
for this so we
can
continue
to
brainstorm...
On Sunday,
March 16,
2014
1:53
PM,
Joe
Schaefer
<joe_schae...@yahoo.com>
wrote:
On
Sunday, March
16,
2014
1:40
PM,
Jake
Farrell
<jfarr...@apache.org>
wrote:
Hey Joe
Thanks
for
reaching
out, we
chose
to
go
with a site
generator
over the
Apache
CMS due to
it
initially
not
meeting
all of
our
needs.
Our
current
needs
for the site
are
-
Markdown to html
conversion
That
one's easy-
its
the
default
for
the
cms.
- Global
variables/config
usable
throughout
in a
template
based
layout
Easy too-
just create
a
perl
hash
somewhere and
make it
available
to
your views.
Code your
views to
pass
that
hash
along to
your
templates
and you
are
all set.
Note if your
hash
contains
objects
you can
make
method
calls
on
them in
your
templates.
Note:
while I
wouldn't
recommend this in
general,
for
smaller
projects
like thrift
that
prefer
convenience
over
separation
of
concerns
you can
have the
django
template
engine
do a
pass
over
your
markdown
before
passing it
along to
your actual
template
page,
just
as is
seemingly
common to
other
popular
site
generation
software.
- Syntax
highlighting
Comes with
python's
markdown
support,
but
some
people
prefer
the look
of
javascript
highlighters.
- Easily
include
code
snippets
from
our
source code
base
Statically I
hope. No
idea
how
to
pull
snippets out of
your
git
repo
via the
cms.
-
Ability to
locally
test
before
committing
Belt and
suspenders
with
the
cms- you
can
build
your
changes
before
committing
with the
build
scripts,
browse
your
changes
before
committing
via
the cms
webgui, or
simply
commit
your
changes
and
view
the
staging
site
prior to
live
publication
(which is
a
separate
step).
-Jake
On Sun,
Mar 16,
2014 at
12:44 PM,
Joe
Schaefer
<joe_schae...@yahoo.com>wrote:
As
it so
happens I
am
interested
in
working on
supporting
thrift's
use
case as
a
potential
user of
the
Apache
CMS.
While the CMS
works
well
for
massive
projects
there
are
things it
could do
better at
supporting
for
more
moderate
sized sites
like
thrift's.
I'd
therefore
like
to
engage
you
folks in a
brainstorming session
about what
sorts of
features
you
want
for your
site
and
to find
simple ways of
supporting
those
features for
you.
Thanks.
