On Thursday, 31 December 2015 at 00:04:03 UTC, bachmeier wrote:
It's the process that requires so much overhead that nobody
wants to contribute. I really tried to do so myself, but I'm
busy, and it is senseless that 95% (or more) of the time I
spend on it is wasted due to a system that is flawed from top
to bottom. The only thing that surprises me is that there are
any contributions.
Exactly. I've put my time into ddoc. Let me give you a brief
history of my efforts:
~2010: I had just written this awesome dom.d library and wanted
to document it and release it to the world. I write stuff like:
/// Returns the text in the element. For example, innerText of
<span>foo<span> is "foo" (without quotes)
string innerText();
And it came out mangled because of ddocs embedded HTML "feature"
which doesn't encode the output. I proposed a fix, using ddoc's
existing ESCAPES macro, and wrote the patch for dmd. Embedded
html would be automatically encoded, but you can still define it
in macro definitions, which can be inline with the comment, so it
isn't hard to use when you want it.
It was rejected. Walter didn't see what the problem was and I was
told to just write $(LT)span$(GT)foo$(LT)/span$(GT). Seriously.
Well, that was unacceptable so I decided to take matters into my
own hands. On Feb 20, 2010, I registered dpldocs.info and started
writing my own web service to delver dom.d's docs, encoding the
output correctly, getting the docs out of dmd's JSON output,
using -D -X.
Well, it worked, but it sucked. The JSON output didn't give me
enough information to do cross referencing, so I wrote a little
search engine to paper over it... and that didn't help much.
I moved on to other things, every so often still advocating to
fix ddoc's escaping, but mostly just writing my docs such that
they were readable in the source code and not bothering with
generating them at all. (I'm told my source tends to be fairly
readable anyway and I like to narrate my thoughts a bit in the
comments including of implementation details so they understand
the pros and cons of my decisions.)
At some point around here, I wrote a crappy Javascript table of
contents for the D website (then hosted at digitalmars.com still)
- the infamous "Jump list". This was meant to be a temporary
measure until ddoc got proper table of contents support.
That sucky Javascript hack is still live today.
2011: We had a proposal for a website redesign. Remember this
one? http://arsdnet.net/d-web-site/index.html
The jump list hack garnered a lot of hate. It was a blob of text
that we couldn't read, but ddoc's design made a user-defined
table of contents extremely difficult to implement. A few
attempts were started, but none finished.
I wrote a program, called improveddoc.d, still on the 'net:
http://arsdnet.net/d-web-site/improveddoc.d that post-processed
the generated HTML to create a cheat-sheet table.
This is what it made of std.algorithm at the time:
http://arsdnet.net/d-web-site/std_algorithm.html
I offered to integrate it into the website tools... and got my
first look at the horror that is that build process.
Nevertheless, running a little post process script to run that on
the html was doable. It also would read tag macros and use them
to organize the content.
The idea (and working program) was rejected because the team felt
a post-processor was the wrong way to do it.
Instead, Andrei manually wrote a std.algorithm cheat sheet and we
added a bullet between the links in the jump list hack. Both of
these have remained basically unchanged to this day and dmd still
cannot generate a table of contents in ddoc.
We did get a website redesign after that,
d-programming-language.org was launched, and Vladimir's forum got
integrated (I wrote one too but his was so much better than mine
that I gladly yielded). ddoc, however, barely changed.
For the next couple years, despite ddoc being unusable for half
my libraries because they are web libs and showing HTML examples
was practically impossible, I still was basically on its side,
while still working on improving the json support. I had tried
the json thing and found the compiler approach to be better, but
the json output is good for a few things and Ddoc's syntax isn't
all that bad if you aim for source readability and clever use of
recursive macros can do some pretty cool things.
I put my support behind dmd. With the vibe.d team also pushing
for improved json support, a few things happened there... though
little of consequence. My opinion of the json output hasn't
significantly changed since 2010: cool for a few things, I like
that we have it, but it is not a substitute for
compiler-integrated ddoc.
We got another website redesign in these years, and moved to
dlang.org. The build process finally got documented after I and
others complained for a while, but it was never actually cleaned
up.
At the end of 2014, yea, even a year before this writing, people
were complaining about ddoc's syntax again and asking for some
Markdown variant to be introduced. Seeing an opportunity to
revive my old encoding problem and finally write some ddoc for
myself, I jumped on the `code` syntax and implemented it,
including proper encoding of the output. Finally, I can write
`<span>foo</span>` and have it legible to the user!
And with it also alleviating the pressure for syntax sugar from
the community... this patch was actually accepted this time! For
the first time in five years, I was excited about ddoc again.
I started using it to write This Week in D (and noticing the
limits the macro syntax place on legible prose source...) and
wrote at some length about my simpledisplay.d, cgi.d, and others
(though not really much on dom.d, ironically. I still haven't
gotten around to updating that.)
Just a few months ago, I got the simpledisplay docs looking
pretty good with ddoc: http://arsdnet.net/arsd/simpledisplay.html
2015 was a good year for ddoc and I, we basically got along,
despite its warts.
Then, October rolls around. A friend of a friend is looking for
work and I say maybe you can help me with some D stuff. Don't
know D? Not a problem, it is easy!
It wasn't easy. From installation, problems arose. I'd then show
how to fix it and explain why.
She'd say "I understand now, but why don't the docs just say what
you just said?"
Good question. I decide to start editing it in... and quickly hit
a brick wall that kills my momentum. I couldn't get through the
nonsense before having to get back to work.
I handle hundreds of D support requests in a year. Sometimes, I
spend so much time on the chat, the learn forum, or Stack
Overflow that I feel like I'm a full-time D support engineer.
Time and time again, people ask things that ought to be pretty
easy to find, yet they ask us anyway.
Maybe they just don't like searching and ask first. I actually
don't mind that - I tell my coworkers that if their attempts at
Google don't yield results in one minute, send me an IM or email
(and continue looking). There's a good chance I'll have an answer
in my brain somewhere, or at least a helpful pointer, and I don't
mind the interruptions at all (if I didn't want them, I'd just
close the chat client!). Better to ping me than waste hours on a
fruitless search.
But still, some people say "I tried searching and couldn't find
it"... and I totally believe that with D. I used it think it was
just because we were young, but it has now been almost SIX YEARS
since my first dive into the website issues and it has barely
changed. There must be something systemic.
Fast-forward to December. We have that base64 question. I go to
edit the site and get stopped by the process again. Nevertheless,
I blamed the makefile and saw new documentation Andrei wrote on
this. Maybe there's still hope.
Then another friend of mine tells me he finally started looking
at the D Cookbook I gave him last year and is writing some code.
I watch him work through a couple problems... and notice that the
website was only useful to him insofar as he could copy/paste
examples. As soon as he modified them and got an error message or
tried to put functions together without pre-written examples,
he'd slow way down. He'd get through it but not easily.
The D leaders know how important examples are. We are often told
adding more is low hanging fruit. I completely agree. But that's
not ALL we need. He wants examples to get started, yes, but he
also wants understanding to go beyond examples. That's where text
helps. That's where understanding the function signatures help.
Eventually, when people go to write their own libraries, they
might want to do Phobos style genericity. You can't do that
without understanding the function signatures... and those blobs
of text are not understandable.
Shift to the forum. We got talking about website redesign on the
forum again a couple weeks ago and I remark how I think just
whitespace formatting of the constraints would help a lot and
almost went in dmd to implement it.
I just got sidetracked doing a quick paragraph fix for Andrei...
and then wanted to see how far I could go. I got it working,
despite the obstacles that ddoc's freeform macros bring, and
surprise, it was rejected. Even the simpler patch that just
collapses blank lines sits unanswered. A tool to look for broken
links sits unanswered. There's always a call to contributions,
but when you do push through the painful process to get the code
up (and the tester, the f$^%$^ing tester), nobody seems to care.
The combined experience over these last six years tells me that
the website sucks and it keeps sucking because changing it sucks
even more than using it. A new strategy was required.
And here we are.
I still need to figure out how I want to add new docs and link
them all in, but I guarantee you it won't consist of editing SRCS
and MANIFEST and posix.mak and win32.mak and std.ddoc and
latex.ddoc and <voice trails off>...
