Re: Mockup of my doc dream ideas
Adam, I wonder could we also have information since what Phobos version particular symbol is available from. For an example, from which version of Phobos we have findSkip() ??
Re: Mockup of my doc dream ideas
On Tuesday, 29 December 2015 at 14:13:54 UTC, Dejan Lekic wrote: Adam, I wonder could we also have information since what Phobos version particular symbol is available from. For an example, from which version of Phobos we have findSkip() ?? Yeah, I was thinking about that too. The info isn't in the doc itself, but could be determined when I update the site here by comparing before and after docs. So it could automatically say "in the .70 update, I found this function that wasn't there before, so I can add a note `Since phobos 2.70`". That will take an additional pass and a cross-version database to query during doc builds but it is totally doable. BTW ddoc also has "Version:" and "History:" sections in the spec we could use to write up changes ourselves...
Re: Mockup of my doc dream ideas
I've started some parsing of ddoc sections and continue work on the formatter: http://dpldocs.info/experimental-docs/std.algorithm.searching.findSplit.html I'm pretty happy with how this is turning out. Going to add more rich info to params later - I realize I can't parse all constraints, but some of them I can - and the ones I can parse cover a *lot* of ground in Phobos! With any luck, by the end of the day, the params there will say something like: haystack Type: R1, forward range The range to search. needle Type: R2, forward range What to look for. Where the "forward range" text will be a link to a conceptual overview of it. Then I will also do automatic See_also: entries that talk about language features it discovered. This one has an auto return value. I'm not sure my generator can detect this reliably yet but I'm pretty sure it is safe to link to a little doc explaining voldemort ranges too here. BTW: pred should prolly explicitly document that it requires the function to return bool.
Re: Mockup of my doc dream ideas
On Saturday, 26 December 2015 at 19:56:13 UTC, Adam D. Ruppe wrote: Still quite a bit to do but it is getting there. I'm using Brian Schott's libdparse to get the source. I want to borrow some of his DCD code to do the scope name lookups and make those dummy links real, and then it is the task of actually formatting the comments and, of course, making sure everything goes through correctly. The name lookup code is here: https://github.com/Hackerpilot/dsymbol. If you haven't seen it already, I recommend that you take a look at this: https://github.com/economicmodeling/harbored
Re: Mockup of my doc dream ideas
On Saturday, 26 December 2015 at 19:56:13 UTC, Adam D. Ruppe wrote: http://dpldocs.info/experimental-docs/std.algorithm.searching.findSkip.html BTW the missing code sample is a documented unittest, which my thing hasn't implemented yet.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 05:06:47 UTC, Adam D. Ruppe wrote: My mock: http://arsdnet.net/mydoc.html Verily, this is the status quo: http://dlang.org/phobos/std_algorithm_searching.html#findSkip Well, the mock is starting to become a reality: http://dpldocs.info/experimental-docs/std.algorithm.searching.findSkip.html Though I might be forced to implement ddoc macros. Ugh. http://dpldocs.info/experimental-docs/std.algorithm.searching.findSkip.html I also ran my simpledisplay.d through it which also has classes and uses ddoc macros much more sparingly: http://dpldocs.info/experimental-docs/simpledisplay.html http://dpldocs.info/experimental-docs/simpledisplay.SimpleWindow.html And this is a test function: http://dpldocs.info/experimental-docs/arsd.doc.test.test.html Still quite a bit to do but it is getting there. I'm using Brian Schott's libdparse to get the source. I want to borrow some of his DCD code to do the scope name lookups and make those dummy links real, and then it is the task of actually formatting the comments and, of course, making sure everything goes through correctly.
Re: Mockup of my doc dream ideas
On Saturday, 26 December 2015 at 20:43:18 UTC, Brian Schott wrote: The name lookup code is here: https://github.com/Hackerpilot/dsymbol. Aye, I'm still just working on figuring out how to use it :) But since DCD does I should be able to borrow a lil code from there to make it happen. If you haven't seen it already, I recommend that you take a look at this: https://github.com/economicmodeling/harbored Actually, while I have heard of it before, I forgot all about it here. Probably would have been a nicer starting point for this than the dscanner modules I used as guides into dparse. Oh well. I might steal some from it now! But I still want to do things a little differently. See, I kinda sorta like ddoc and it isn't *fatally* flawed... but it is *fundamentally* flawed and I actually want to get away from it, not implement it again. I want to have a slightly longer summary. I want to have semantic, properly encoded output. I want pasted links to magically become tags and rich automatic linking. I do NOT want arbitrary foo: strings at the beginning of a line to start a new section. I do NOT want the current symbol to be highlighted. I do NOT want $1 appearing in the text to do weird things. So that's going to put a limit on my compatibility with Phobos. Though maybe, just maybe, I'll use your libddoc on modules that start with std. and core. then do my own things on my modules.
Re: Mockup of my doc dream ideas
On Saturday, 26 December 2015 at 21:38:23 UTC, Adam D. Ruppe wrote: I do NOT want the current symbol to be highlighted. Harbored doesn't do that either. The only time I use that feature in ddoc is to disable it.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 05:06:47 UTC, Adam D. Ruppe wrote: I wrote this manually as a mock of what I really want the docs to look like. I only marked up the top box and a wee bit of the bottom. The body text of the doc is written by us and is OK, but the function signature is a mess. [...] Why not take it to the next level while youre at it? Add user comments that can be rated by users and sorted by date.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 14:13:56 UTC, Adam D. Ruppe wrote: On Friday, 25 December 2015 at 09:18:45 UTC, Israel wrote: Why not take it to the next level while youre at it? Add user comments that can be rated by users and sorted by date. My dream does not include user comments. I don't think well-written documentation benefits from them... What happens when I see a DOC with comments is that sometimes the comments are more clear than the Doc itself, or there are tips or tricks that was not "well" documented. Bubba.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 09:18:45 UTC, Israel wrote: Why not take it to the next level while youre at it? Add user comments that can be rated by users and sorted by date. My dream does not include user comments. I don't think well-written documentation benefits from them, and adding such a thing would surely include some kind of speed hit on the page, harm downloadability, etc.; the page would then have a dynamically loaded section. That isn't necessarily a dealbreaker, it just isn't a part of my dream. On the other hand, I do dream of a suggestion box on the page that contacts the authors so they can incorporate it... which isn't totally different than comments. So I'll put it down as a maybe.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 14:50:06 UTC, Adam D. Ruppe wrote: This linked page is my dream for a narrow part of the docs: the function signature. The whitespace formatting can be worked into existing ddoc (I'm angry with existing ddoc and don't want to work with it for a while, but it isn't hugely difficult to do and I think will be a worthwhile incremental improvement), but my other vision is more revolutionary: docs that are written from a "how do I do this" viewpoint rather than a "how do I use this". Aren't these usually called tutorials? Or am I misunderstanding what you mean here? My ideal system would stay one step ahead of me or allow me to stay on the same page as often as possible. Yes. I'm actually a wee bit split on the one page thing: I like having a long, single page with all the info. I just think there's a *lot* of advantages to it and actually consider this a strength of dlang.org (well, until std.algorithm got split up into separate modules. Now it is a pain, and I updated my dpldocs.info search engine in response) This is literally one of the things that bothers me most about the documentation. Mostly when I try figure out how to do something I use google. Usually google picks out a couple of std module documentation pages for search queries I enter and then I have to Ctrl+F-guess my way through pages that make my scrollbar almost vanish or guess from function names (which are often somewhat abstract so it's hard to guess at what functions you should really check out). Then when you try to figure out if a function does what you want it to or not, often examples are missing and reasoning how to use it if it involves templates and template constraints isn't always straightforward if you aren't too familiar with the language and its idioms (ie: if you're new to it and trying to learn). A single page really detailing what a function does and providing an example for how to use it is what I very much prefer. I may also be a bit odd in this regard, since I use a pretty high zoom level in my browser, so scrolling down long pages (or navigations, such as the module list of std) is probably much more annoying to me than to everyone else (I often also want to hop through the documentation, like "oh okay I get how to use this function, but lets also check these two other things while Im here *scroll scroll scroll scroll*"). Plus I'm probably biased since I'm coming from C# and thus am heavily used to the MSDN docs. Thought, one page per function is ok at times too. :-)
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 15:51:02 UTC, rcorre wrote: On Friday, 25 December 2015 at 14:24:34 UTC, Bubbasaur wrote: What happens when I see a DOC with comments is that sometimes the comments are more clear than the Doc itself, or there are tips or tricks that was not "well" documented. Bubba. If the comments are more clear than the Doc, then they should _be_ the doc. Similarly, useful tips and tricks should probably be included as doc examples. In other words, I think we should encourage contribution to official documentation rather than dividing between official docs and user comments. This does raise the bar to entry though, so maybe a suggestion box could be a nice middle ground. I agree, but you know in a perfect world nothing always work as we imagine, but anyway I think the suggestion box is a good idea. Bubba.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 07:41:11 UTC, James Hofmann wrote: On Friday, 25 December 2015 at 05:06:47 UTC, Adam D. Ruppe wrote: I strongly agree Markdown is simple to use, and well supported. No need to do work that has already been done. Besides, Github is pretty popular nowadays. assert(false, "jQuery has a really good documentation.");
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 14:24:34 UTC, Bubbasaur wrote: On Friday, 25 December 2015 at 14:13:56 UTC, Adam D. Ruppe wrote: On Friday, 25 December 2015 at 09:18:45 UTC, Israel wrote: Why not take it to the next level while youre at it? Add user comments that can be rated by users and sorted by date. My dream does not include user comments. I don't think well-written documentation benefits from them... What happens when I see a DOC with comments is that sometimes the comments are more clear than the Doc itself, or there are tips or tricks that was not "well" documented. Bubba. Just one example that I said early, if you look here: https://dlang.org/dmd-windows.html#switch-L (-Llinkerflag) It's written: "pass linkerflag to the linker link.exe , for example, -L/ma/li". But at least on Windows, you need to put a space between -L and the PATH. Which It's weird, since with "-I" flag you don't need any space. It took me 30 minutes until I find why my program wasn't compiling. Thanks to the Forum. Bubba.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 07:41:11 UTC, James Hofmann wrote: 1. There's more code than ever and more of a need for code to be well-documented on a conceptual/hand-holding level Absolutely. general idea of fitting the text into a certain container is similar to the API doc tradition of following the class-and-method structure of the code. I recently went to add some explanation to std.base64 on how to get a string out of it since someone asked here. But I actually stopped on the official website just because I couldn't find a place to put it. It didn't really fit in base64.d, the existing structure was all wrong and it was the wrong level. You can apply the concept using base64 but it isn't specific to it. It didn't fit on any of the existing pages. And adding a new page was just too painful for me. I ended up posting a couple quick sentences to the wiki - not even linked from anywhere - then running away. Everybody lost just because I felt blocked by the existing structure. (and I don't know wiki syntax, I felt like that was another blocker, and the dlang.org intralinking situation is terrible too) This linked page is my dream for a narrow part of the docs: the function signature. The whitespace formatting can be worked into existing ddoc (I'm angry with existing ddoc and don't want to work with it for a while, but it isn't hugely difficult to do and I think will be a worthwhile incremental improvement), but my other vision is more revolutionary: docs that are written from a "how do I do this" viewpoint rather than a "how do I use this". We still have the "how do I use this" sections, the existing pages basically fit that. But they feel written bottom-up. I want to do top down. I've said before that I currently have a contractor working on some docs. My instructions for her were simply "write something cool in D and document it for future newbies". What she's written so far is already very different than what we have, though there's still a lot of work to do. (In particular, I'm kinda struggling with how much editing I want to do. The point is to document a newbie's journey as a guide to other newbies but I also see her making a few mistakes that I kinda want to correct... but she's learning quite a bit from those mistakes too and so could the readers. I hate tutorials that assume everything works. Users need to get used to seeing error messages and understanding both what they mean and why they are there.) Once we get started, other things will follow. A friend of mine who is new to D just wrote something to save text to a google doc this week. The result is useful to him and his work and while it isn't super advanced D, the journey would be a good experience for others too. What libraries did he use? How did he overcome problems? From these results-driven narratives, I can go back and start linking in more things - related narratives (generic D REST api tutorial, JSON use), language feature discussions (what was that nested function used in the callback? why use `is null` instead of `==`? what's `opIn`?) and idioms, concepts like string encoding and type system wrangling, and, of course, library docs for the used pieces. Then link it back to the top level. Of course, this stuff is a lot of work. I only have so much money I can spend on hiring people to work with this for me, and I have even less time to spend writing it myself... which is one reason why I'm now working on my new automation code. The mockup is good in that it cleans up the messy look and feel of the current presentation, and suggests the potential for interactivity. My main concern from a presentation standpoint is high latency, which in these interactive help systems typically comes in the form of slow page loads every time I try to click on something. That's an issue of implementation. I'm literally in the same room as the server TWiD is on so I can't objectively judge its speed, but try clicking around the issues here: http://arsdnet.net/this-week-in-d/dec-20.html does it respond reasonably well? I made very little effort on optimization there... but also kept it simple and did all the processing ahead of time. The server hands out pre-built .html.gz files on demand, no dynamic parts. My D doc search engine has dynamic parts: http://dpldocs.info/findSkip though I also wrote it with a bit of an eye toward efficiency because waiting for docs drives me nuts. The static parts can all be downloaded in a bundle easily enough. Search could be a downloaded program too. Offline docs matter to me - I'm not always on the internet either! My ideal system would stay one step ahead of me or allow me to stay on the same page as often as possible. Yes. I'm actually a wee bit split on the one page thing: I like having a long, single page with all the info. I just think there's a *lot* of advantages to it and actually consider this a
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 14:24:34 UTC, Bubbasaur wrote: What happens when I see a DOC with comments is that sometimes the comments are more clear than the Doc itself, or there are tips or tricks that was not "well" documented. Bubba. If the comments are more clear than the Doc, then they should _be_ the doc. Similarly, useful tips and tricks should probably be included as doc examples. In other words, I think we should encourage contribution to official documentation rather than dividing between official docs and user comments. This does raise the bar to entry though, so maybe a suggestion box could be a nice middle ground.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 22:26:46 UTC, bachmeier wrote: Filing a bug is a better approach. It's best to keep everything in one place. Bugzilla sucks hard though, on pretty much every level. It is separate from the page itself, people might not even know it is there, you have to log in, and then fill in a bunch of stuff manually. On the maintainer side, it is really hard to search, a pain to find new issues that are relevant to you, and it is just separate again from the PR process and all that. The suggestion box, on the other hand, would immediately alert me, it would tell me what page the suggestion came from, and the user doesn't have to jump through many hoops. Then, I can manage the backend however I like too. Heck, it could even open a bugzilla issue.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 17:00:05 UTC, default0 wrote: On Friday, 25 December 2015 at 14:50:06 UTC, Adam D. Ruppe wrote: Aren't these usually called tutorials? Or am I misunderstanding what you mean here? A single page really detailing what a function does and providing an example for how to use it is what I very much prefer. Plus I'm probably biased since I'm coming from C# and thus am heavily used to the MSDN docs. Thought, one page per function is ok at times too. :-) This is exactly how i feel. You might see me as coming too weak because im used to C# and MSDN Docs holding my hand but with the help of those docs and autocompletion, i learned a decent amount of C# in less than 2 weeks without rarely asking anyone for help. This isnt something you could say for C/C++ or even ruby and python.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 15:51:02 UTC, rcorre wrote: On Friday, 25 December 2015 at 14:24:34 UTC, Bubbasaur wrote: What happens when I see a DOC with comments is that sometimes the comments are more clear than the Doc itself, or there are tips or tricks that was not "well" documented. Bubba. If the comments are more clear than the Doc, then they should _be_ the doc. I agree with this. The PHP documentation is a mess because of the comments. Similarly, useful tips and tricks should probably be included as doc examples. In other words, I think we should encourage contribution to official documentation rather than dividing between official docs and user comments. I think there should be link to the wiki for tips and tricks and more examples. Sometimes you don't want 10 examples in the official docs, even if they will help some users. The main reason for using the wiki would be specialized examples. This does raise the bar to entry though, so maybe a suggestion box could be a nice middle ground. Filing a bug is a better approach. It's best to keep everything in one place.
Re: Mockup of my doc dream ideas
On Saturday, 26 December 2015 at 03:01:25 UTC, Israel wrote: This is exactly how i feel. You might see me as coming too weak because im used to C# and MSDN Docs holding my hand If it wasn't for Microsoft documentation (I didn't use MSDN per se, but a downloaded win32.hlp file - I didn't have the Internet at home until the last decade - from the Windows 95 era with basically the same content), I'm not sure I'd know half the stuff I know now.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 19:00:13 UTC, karabuta wrote: I strongly agree Markdown is simple to use, and well supported. I don't like Markdown personally, though I don't hate it either. A couple features in it are cool, but most of them are just meh to me and a few of them I actively dislike. However, the syntax the documentation is written in is of less relative importance to me than the content and the layout. I write ddoc every week and it annoys me a little... and I also think it has fundamental design flaws that put a hard limit on how much it can improve... but it is basically good enough. I also mostly like writing XML, it really isn't bad! But you'll notice the "dream doc" mock still had ddoc. Even if I were to change things, I'd change it for my new docs... I wouldn't want to go through and convert all of Phobos. (Besides, I'm not the only consumer of the phobos source.)
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 17:00:05 UTC, default0 wrote: Aren't these usually called tutorials? Or am I misunderstanding what you mean here? Oh maybe, I've heard "tutorial" used in a lot of contexts and a lot of meanings though, so I wanted to be more specific. This is literally one of the things that bothers me most about the documentation. Mostly when I try figure out how to do something I use google. Google does a really bad job on the D docs... and the D docs don't have enough anchors for it to pick up anyway. (Even if I want to manually share a link, there aren't links to most the ddoc sections! Aaargh!) But the improved interlinking ideas will help with all of that, single pages or separate pages. templates and template constraints isn't always straightforward if you aren't too familiar with the language and its idioms (ie: if you're new to it and trying to learn). Yeah, I think there's three general levels of users: 1) those who don't speak D-ese at all and need to learn the language. D-ese includes more than D itself, but also phobos idioms, general layout, etc. These people really need higher level docs to make sense of it. Tutorials and conceptual overviews need to be written with these people in mind. 2) People who are conversational, but not fluent, in D-ese. They can basically get stuff done but don't know it all. The function-level docs should be good enough for them, but it should translate some of the harder stuff to plain English. Don't ask them to read a constraint mess, just say it in words and in examples. This is the target audience of the stdlib inline docs IMO. They know how to get there, but then need some assistance bringing it back home. 3) People who are fluent in D-ese. The docs aren't terribly important to them except to quickly refresh themselves. They are fully capable of reading the source, but like the docs for a quick outline. The actual content isn't really important to these folks, they just want a convenient index. Plus I'm probably biased since I'm coming from C# and thus am heavily used to the MSDN docs. Yeah, MSDN is great, I think it will be my main model.
Re: Mockup of my doc dream ideas
On Friday, 25 December 2015 at 05:06:47 UTC, Adam D. Ruppe wrote: Next, observe the highlighted words... and go ahead and try to click on them. Oh snap! They are links to the language features used. This is the web, let's link all over the place. I want to have relevant conceptual overviews available one click away on every function. Before I get to remarks about this demo, I'd like to connect the D doc situation with the current state of technical writing as I understand it(it's something I'm moving towards professionally). There's a set of trends I've identified in my survey of the landscape: 1. There's more code than ever and more of a need for code to be well-documented on a conceptual/hand-holding level, as well as the minimal "turn comments into docs" situation. "User manuals" are out - everyone wants easy UX - but "API docs" are in. 2. A shift away from proprietary tools like Word or Adobe Framemaker towards a heterogenous, open-format ecosystem - e.g. conversion tools like pandoc are important now. 3. "Topic-based authoring". This is mainly led by the DITA format, which imposes a particular *model* for how the documentation is laid out and how pieces of information may be related, vs. the more free-form linkage of a wiki or the linear text of traditional books. You typically see these topic-based systems appear in big enterprisey application software, but the general idea of fitting the text into a certain container is similar to the API doc tradition of following the class-and-method structure of the code. 4. A default approach that uses lighter markup like Markdown or wiki text in order to minimize the barriers to entry. "Heavy" formats like DocBook add friction to the prose-writing process, even though they're necessary once you pile on enough features. Authors may write in one and convert to the other. The ideal documentation system for D addresses each of these - dedicated authors who are primarily adding prose text, a variety of output formats, models for defining code/data/textual relationships, and low friction inputs. The mockup is good in that it cleans up the messy look and feel of the current presentation, and suggests the potential for interactivity. My main concern from a presentation standpoint is high latency, which in these interactive help systems typically comes in the form of slow page loads every time I try to click on something. My ideal system would stay one step ahead of me or allow me to stay on the same page as often as possible. Options for offline docs also usually fix the problem. There is also a pretty good conceptual base in the existing ddoc system w/r to its goal of extending documentation comments. Its defaults are biased towards HTML presentation, which is OK, albeit not as trendy as Markdown. What ddoc misses, though, is strong capability to invert the semantics towards prose-first - for example, if I wanted to write a very extensive tutorial that discusses a large codebase or how the codebase changes over time, then I'd write prose first, but reference an annotated version of the codebase so that I can call out specific parts of it. This is something that could be realized with a crude, hand-rolled system to extract the code and inline it, but it would most likely give up the semantic possibilities; I'd end up only having some syntax highlighting and not be able to let the reader click on the source and see the file in context, link function name callouts to their API documentation, etc. One way to start seriously addressing all of these various issues is to export an annotated, pre-highlighted D source file with metadata, which could then be slurped up and processed again into an interactive API doc, a prose text, presentation slides, etc. This work can be seen as a two-parter: parsing the code and emitting useful, easily processed data - and then making default presentation of that data good, so that minimal effort is still rewarded. It's conceptually similar to the problem space of IDE integration, where different IDEs may want to query the source in different ways. Semantic documentation source is also one of the rare situations where the XML ecosystem is at its best - XSLT basically exists to declaratively transform semantic markup into formatted presentation, as is done, for example, with DocBook. It wouldn't solve any *particular* problem to target XML, but it would expand people's ability to dream up and implement presentation-side solutions.
