Re: working on the dlang.org website
Better late than never (the last couple weeks have been very busy): On Tuesday, 23 July 2013 at 21:00:08 UTC, Walter Bright wrote: It's true that they are based on HTML output. However, and this is a big however, they need significantly different HTML output than one puts on a web site. This is currently accomplished by changing the macro definitions that Ddoc uses, and by carefully recoding the Ddoc source to use those macros. While generating ebooks is often billed as "just pipe your website HTML through our converter program!" the reality is that you'll get more or less utter garbage if you try that. I've published several ebooks that also exist as web pages, so I'm familiar with the process. I don't entirely agree with this, although I understand where you're coming from. The cost/benefit of settling this firmly favours the costs, so I'll let it slide. I assume this one is it? https://github.com/D-Programming-Language/dlang.org/pull/320 I posted a response. (But in general, PR's that are flagged as "We can’t automatically merge this pull request" tend to not get much attention. Despite that, we can and should do better.) Can you please make some PR's which illustrate your work in converting it to HTML 5? Well, I can't, as I've said a few times, because I was instructed to wait for https://github.com/D-Programming-Language/dlang.org/pull/271 to get merged, which, I'm told, will shorten my workload considerably. I realise, since complaining about it, that you've commented on that PR. Maybe, if I have some time, I'll try to tackle it so that it'll automerge (which is tricky because merges are a moving target). http://forum.dlang.org/post/ypcykidvradkrhnob...@forum.dlang.org has quite a few responses; more than most threads. But I'm afraid that's analogous to saying "Whilst it is true that we haven't come around to fix your collapsed roof, we've picked up when you called more often than we have for other customers, so I don't see what you're upset about." The arguments were (I thought) well laid out in that thread and responded to. Reasonable people can disagree - it doesn't mean that one side is irrational. My training is in business, not computer science. One learns in business school that the successful companies satisfy the needs of a particular market. When the customers' needs drift from what the business produces - as invariably happens in almost all markets - the business can either adapt to its changing environment or invite competitors to do its job for them. Therefore, regardless of how irrational your customers are, there is often a benefit to giving them what they want.
Re: working on the dlang.org website
On Sunday, 14 July 2013 at 20:35:45 UTC, Walter Bright wrote: 3. HTML, PDF, Ebook, and CHM outputs are generated from Ddoc. Walter, with respect, I know you're too smart to be saying something silly like this. Surely you know that ebooks and CHM are specially-compiled HTML files. To imply that DDoc, and not HTML, is the common denominator between these outputs brings me to the acme of frustration as I have said over and over again that, at the very least, ebooks depend upon good HTML output. To this end, I have tried for months to help bring dlang.org up to HTML 5 standard if, for no other end, to be able to compile the spec into an epub format. I even toed the github waters by submitting a trivially simple PR which, for the past months, has gone completely ignored. In the meantime, my offers on help with other dlang.org PRs have also gone ignored. The thread in which I keep offering to help (Dlang spec rewrite) has gone ignored. To argue that contributions are not languishing or that volunteers are not being ignored necessarily means that I do not exist. If I have learnt one thing in the past few months, it is that any attempt to dispute using DDoc to program an entire website is a fool's exercise. Despite vocal objections from the plebian masses, the DDoc architects and maintainers will stubbornly defend its usefulness. One can hardly blame them: wouldn't any of us defend code that we had carefully designed or write? Therefore, dlang.org will stay, for better or worse, in DDoc. It's not worth arguing. As for willing contributors, it seems to me that the maintainers have an agenda to which they are adhering. Contributions focussed on other areas are diversions. Have I said anything that is factually incorrect?
Re: DLang Spec rewrite (?)
Ping! I'm just bumping this thread to see where the status of integrating pull request 271 is and whether there's anything I can do to expedite matters. I've noticed that there are some changes to dlang.org's website source. Are these changes working towards HTML 5 compliance? (or, at least, the part of HTML 5 that probably won't change).
Re: DLang Spec rewrite (?)
On Saturday, 29 June 2013 at 11:33:16 UTC, Daniel Murphy wrote: To be honest, you just have to keep bugging people. I mostly review compiler pulls, and I am much much more likely to review something that shows up in my inbox than something that sits patiently in the list. If you make enough noise somebody will eventually reply. Sigh, I know. I just don't want to get on anybody's bad side. Maybe I can do that just by keeping this bumped...
Re: DLang Spec rewrite (?)
On Monday, 27 May 2013 at 02:11:00 UTC, Andrei Alexandrescu wrote: I think it would be great. In particular, an ebook format would be good. You may want to wait until https://github.com/D-Programming-Language/dlang.org/pull/271 is in. It systematizes macros a lot and it may offer answers to many of your questions. Andrei I would still like to work on compiling the DLangSpec into HTML5, but I've noticed that pull request 271 hasn't been touched in over 4 months. Further, I sent in a pull request to move the DLangSpec source files into their own folders and haven't gotten so much as a 'worst pull request ever' in response. I fully appreciate that people are very busy - including me - so I want to know if there's anything I can do to help things along. At least with respect to pull request 271, is there anything that I can do to help get it merged into master so I can get working on the HTML5 DDoc?
Re: DLang Spec rewrite (?)
On Monday, 27 May 2013 at 02:11:00 UTC, Andrei Alexandrescu wrote: You may want to wait until https://github.com/D-Programming-Language/dlang.org/pull/271 is in. It systematizes macros a lot and it may offer answers to many of your questions. Andrei Professor, what sort of feedback would help get that pull expedited through? I downloaded it, and it compiled for me, but beyond that I don't know what to look for.
Re: DLang Spec rewrite (?)
Yep, and that seems like a bad idea, so I'll just update the macros is the xhtml.ddoc file
Re: dlang.org/posix.mak dependencies
On Tuesday, 28 May 2013 at 00:50:04 UTC, Jonathan M Davis wrote: On Tuesday, May 28, 2013 02:14:06 Borden wrote: Good evening, all, I'm making a new thread on this because my question isn't strictly related to the DDoc issues I've mentioned in earlier threads. Rather, it has to do with the dependencies of the posix.mak file in the dlang.org repo (on GitHub, for greater clarity): posix.mak, beginning at line 236, compiles local copies of DMD, DRuntime and Phobos. Is all of this extra compilation really necessary for someone who, presumably, already installed D using the installer packages? Wouldn't it be more reasonable to see if the most current version of DMD is installed and, if not, panic out and tell the user to go and fetch the latest copy from the website? Then the rest of the makefile could use the installed copies on the user's computer rather than download and recompile them? This question is specific to the dlang.org repo and NOT to any other repos hosted at GitHub. It's a bad idea to assume that a released version of dmd and Phobos will work with the current git repos. It's far less of an issue than it used to be, but breaking changes still happen (some of which are regressions and get fixed and some of which are on purpose), and there's no guarantee that there haven't been changes which require that you have the latest of everything in order to build the git versions. - Jonathan M Davis Good evening, Jonathan, The thing, though, is that, unless I've misread the makefile, dlang.org doesn't have any code that needs compiling. All it needs DMD for is to generate the documentation (which, I'm presuming, can be accomplished by the most current released version) and generate the docfiles for the Phobos libraries (which, I'm guessing, should be compiled against a current release of the libraries).
Re: DMD source violates c++11 standards.
On Tuesday, 28 May 2013 at 00:12:57 UTC, Walter Bright wrote: On 5/27/2013 4:47 PM, Daniel Murphy wrote: "Hans W. Uhlig" wrote in message news:idtvbddofuxwpsbto...@forum.dlang.org... This also makes compiling LDC with clang rather difficult DMD is not written in C++11. Turning of the C++11 warnings in clang will fix this problem. Also, you can submit a bugzilla entry for it, or even a pull request. There's no reason for such code in DMD to remain. I was thinking about that: surely D's mature enough that dmd can be written in D?
dlang.org/posix.mak dependencies
Good evening, all, I'm making a new thread on this because my question isn't strictly related to the DDoc issues I've mentioned in earlier threads. Rather, it has to do with the dependencies of the posix.mak file in the dlang.org repo (on GitHub, for greater clarity): posix.mak, beginning at line 236, compiles local copies of DMD, DRuntime and Phobos. Is all of this extra compilation really necessary for someone who, presumably, already installed D using the installer packages? Wouldn't it be more reasonable to see if the most current version of DMD is installed and, if not, panic out and tell the user to go and fetch the latest copy from the website? Then the rest of the makefile could use the installed copies on the user's computer rather than download and recompile them? This question is specific to the dlang.org repo and NOT to any other repos hosted at GitHub.
Re: DLang Spec rewrite (?)
Oh, and another thing: XHTML adopts the XML practice of only defining the lt, gt and amp entities and no others (like nbsp, mdash, accented, or non-Latin characters). Since Unicode is, by and large, universal, I've read that the recommended practice for including characters not on a standard US keyboard is to copy them from a character map and save the file in a Unicode encoding. I intend to follow this guidance in writing the (x)html.ddoc template. As such, should I keep the existing 'entity' macros or use the Unicode characters in the DLang spec source files? I imagine that Andrei will immediately comment that .tex files are supposed to be in ASCII. Suggestions?
Re: DLang Spec rewrite (?)
On Monday, 27 May 2013 at 03:32:54 UTC, Andrei Alexandrescu wrote: doc.ddoc is the general skeleton file for defining the online documentation. html.ddoc contains HTML-specific macros only, without having anything to do with our site's specific format. For greater clarity, html.ddoc will produce a generic, HTML-compliant file. In contrast, doc.ddoc will add all of the dlang.org-specific decorations and boilerplate? That being the case, would it make more sense for me to upgrade html.ddoc to HTML5 (since it's in candidate rec status over at W3C)?
Re: DLang Spec rewrite (?)
On Monday, 27 May 2013 at 03:32:54 UTC, Andrei Alexandrescu wrote: Yup, you got your work cut for you. Then again, wait til that diff is merged. It fixes a bunch of problems. That's OK. As long as I have some guidance on what to do I should manage. This effort isn't entirely selfless - part of tidying up the DLang spec is to help me learn D, too.
Re: DLang Spec rewrite (?)
On Monday, 27 May 2013 at 02:11:00 UTC, Andrei Alexandrescu wrote: I think it would be great. In particular, an ebook format would be good. You may want to wait until https://github.com/D-Programming-Language/dlang.org/pull/271 is in. It systematizes macros a lot and it may offer answers to many of your questions. Andrei I appreciate the direct answer to my question, Professor. I would start anyway, in my own source copy, checking the existing .ddoc files and updating, in the few places necessary, the tags from HTML4 to HTML5 - most of these changes are to the HEAD section, anyway, and shouldn't require changes. There are two problems that I've already run into, which I'll need experienced help with: 1) doc.ddoc and html.ddoc define many of the macros that I need, but some of them I'll need to redefine for HTML5. Walter's response to how dmd resolves 'macro inheritence' doesn't clarify for me whether I should override the non-HTML5-compliant macros or rewrite the whole file. I hope it's not the latter. Also, I don't understand the difference between doc.ddoc and html.ddoc - what is each file supposed to do, exactly? 2) One I have my xhtml5.ddoc, it won't compile the .dd sources correctly because many of the .dd files aren't written in a manner where simple macro expansion will generate HTML5 compliant code. To solve this, I'll need guidance on how to change the .dd files to get xhtml.ddoc to work without breaking the other files. To this end it would be most helpful to develop a standard list of macros to use in the DLang spec sources and edit the non-conforming .dd files to follow it. It seems right now that the source files define whatever macros they like and leaves the onus on figuring out what each means on the .ddoc files.
Re: DLang Spec rewrite (?)
Before we get too off topic in this thread, is there demand for an xhtml5.ddoc file? If so, I'd like to make some changes to the other DDoc files as to minimise code reuse and minimise ambiguity in 'inherited' macro definitions. I'm willing to put in the time but I can't do it alone. If there's no demand, that's OK, too, and I'll put the matter to rest.
Re: DLang Spec rewrite (?)
Thank you for the suggestions, Juan. For the purposes of generating a single set of XHTML5 documents, your advice would work. What I'm trying to do, however, is update the makefiles for the website source so that ePUB files become a target. I worry, therefore, that pumping the DLang spec through several conversions will give me less control over the resulting output. More importantly, though, it will make the makefiles less portable because anyone who wishes to use them will have to install all of the dependencies, so I'm trying to avoid that. Again, what I'd ideally like to do is write an xhtml5.ddoc file which will give all of the necessary macro definitions to compile the DLang spec into tidy XHTML5 files. Unless I'm mistaken, I don't think that this is a very unreasonable goal. However, I could be wrong and a solution like the one you suggest may be the only way to do this...
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 06:43:46 UTC, Jonathan M Davis wrote: So, in questions of whether ddoc is powerful enough or expressive enough to do something (which appears to be the thrust of Borden's complaints) aren't affected by it. How I'd rewrite DDoc from scratch as its own markup language is not quite what I'm trying to get at in this thread. From what I've gathered from Walter's responses, if I've understood correctly, is that the idea behind DDoc is to provide the simplest rules-based formatting scheme possible for the purposes of generating documentation at the same time one compiles code. I just want to make sure that I understand what I'm working with. My 'complaint' - although I would prefer to have my observations about difficulties working with a markup system be called 'observations' - is that the current body of text files which comprise the DLang spec source cannot be easily compiled into clean, well-formed, XHTML5-compliant files from which I can build an ePUB file. To solve this problem, and based on responses I got to previous related threads, I offered in my first post to translate the DLang spec files into a markup designed for documentation. This idea was promptly refuted as being unwelcome effort as, it was explained, the DDoc spec is written in a way which is both sufficient for its purposes and is independent of any particular markup language. I am willing to keep working with the DDoc macros to try to get them to output the XHTML5 files that I want. However, before I can continue, I need guidance on: a) How I can modify the DLang spec files to enable me to translate them into the HTML5 files that I need; and b) Avoid breaking existing compilation into other formats (such as Latex, PDF, HTML4, etc.) (I apologise if my message came across as hostile. It's rather late where I am and I wanted to get this into the aether before I went to bed. I don't mean any insult if anything I've writen could be interpreted that way)
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 04:57:12 UTC, Borden wrote: On Sunday, 26 May 2013 at 04:30:46 UTC, Walter Bright wrote: Again, this is deliberate. Macros are set up so that the last one overrides all the previous ones, enabling a hierarchy of them using ddoc files. It's a simple form of 'inheritance'. And perhaps this point could be clarified (and, when I next attack the source I'll test it). I have one.ddoc two.ddoc and src.dd. In src.dd, I use $(MY_MACRO x). one.ddoc has the line MY_MACRO=Called one on $1; two.ddoc has the line MY_MACRO=Called two on $1. So, I now run dmd -o- -D one.ddoc two.ddoc src.dd. What does src.html say? and by $1 I mean, of course, $0.
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 04:30:46 UTC, Walter Bright wrote: Again, this is deliberate. Macros are set up so that the last one overrides all the previous ones, enabling a hierarchy of them using ddoc files. It's a simple form of 'inheritance'. And perhaps this point could be clarified (and, when I next attack the source I'll test it). I have one.ddoc two.ddoc and src.dd. In src.dd, I use $(MY_MACRO x). one.ddoc has the line MY_MACRO=Called one on $1; two.ddoc has the line MY_MACRO=Called two on $1. So, I now run dmd -o- -D one.ddoc two.ddoc src.dd. What does src.html say?
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 03:56:08 UTC, Jonathan M Davis wrote: AFAIK, your recent posts on ddoc are the first that anyone has complained about it in quite some time. There are plenty of folks who want various improvements to the online documentation, but doesn't necessarily require doing anything to ddoc, and it's rarely the case that someone complains about ddoc itself. That's fair, and has probably only come up now because I've decided - granted with very little experience in DDoc - to kick the proverbial hornets' nest by diving head-first into it and do crazy things with the source. On Sunday, 26 May 2013 at 03:56:08 UTC, Jonathan M Davis wrote: Normally, you'd nest things by nesting macros. e.g. $(NESTED stuff $(NESTED more stuff $(NESTED yet more stuff) $(NESTED other stuff))) Indeed. I suppose, in addition to my grievances about error checking, is suggesting that DDoc should include parentheses closure checking? On Sunday, 26 May 2013 at 03:56:08 UTC, Jonathan M Davis wrote: However, ddoc should allow you to do pretty much anything that involves simply transforming the content of a macro to somethnig else. The macro takes a set of arguments and then creates something new with them by rearranging them and adding stuff around them and the like. And maybe that's what my biggest frustration with the macros is (or at least how they're implemented in the DLang spec): they read like an abstracted wrapper for HTML, and someone like me immediately yearns for the extra features that got simplified out. Say, for example, we have a $(B) macro, and I want some of them to have ids or classes and others not (since my eventual CSS file will have special formatting rules for them). To add this functionality, I would have to find all of the $(B)s and rewrite them to say $(B id, class, content). For each one that I miss, I'm going to have an empty element with its id set to the content! On Sunday, 26 May 2013 at 03:56:08 UTC, Jonathan M Davis wrote: The exact set of macros used with the online documentation may very well be too specific to html 4, and it may be that the macros will have to be rewritten and moved around in the documentation And now I think we're getting to the heart of the problem. I might have a more favourable opinion of macros if they were more descriptive of the content in the DLang spec source files. That would allow a fair bit more flexibility. Still, I don't think there's any avoiding that any macros requiring nested formatting or special parameters will require them to be written with their intended output formats 'in mind' to work correctly. And, of course, I am very reluctant to attack any of this lest I start breaking the website, Latex or PDF generation in getting the DLang spec 'HTML5 ready'. Hence, why I'm at a standstill.
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 03:51:48 UTC, Walter Bright wrote: It's not totally random. I've designed one macro language before (ABEL), and have implemented 3 (ABEL, Make, and C preprocessor), so I knew what I wanted. Ddoc is very similar to Make's macro system. BTW, the C preprocessor takes the cake for being both horrendously complicated (most implementations take about 10 years to get right) and woefully inadequate. Good evening, Walter, I noticed the similarities to DDoc macros and Make immediately. Again, I think the documentation system you designed is excellent. The 'sugar' I suggested in an earlier post seemed, at least to me, in line with the general 'common sense syntax' that you implemented elsewhere - such as with defining code, variables, dates, authors, paragraphs, etc. Then again, you're dealing with an (aspiring) accountant, not a computer scientist, so I only have experience in trying to make complex things look pretty and not caring about all of that optimisation and implementation stuff! Again, I have nothing but the highest respect for the work you've done. Think of it this way: if I didn't admire D so much, I wouldn't be so determined to get its documentation onto eReaders!
Re: DLang Spec rewrite (?)
Good evening, Professor, On Sunday, 26 May 2013 at 02:05:55 UTC, Andrei Alexandrescu wrote: What vexes me is that all the sugar you propose goes against what you opened with... I'm not trying to cause any offence, and I apologise if any of my phrasing or comments are construed that way. I know that I can be a little bit terse at times but I hope it's taken in the best possible way because I have full respect for the design and implementation of the language. On Sunday, 26 May 2013 at 02:05:55 UTC, Andrei Alexandrescu wrote: I can hypothesize that the shortest path between where we are and what you're trying to accomplish is a few dozens of macro definitions. Did you try doing that and failed? Indeed, it has run into some snags: 1) One of the first problems I ran into was coming up with rules for spec.dd. My original objective was to enclose the $(TOC) macro into tags, consistent both with the HTML5 spec and ePUB3. However, tags are not allowed within the tags, but I also don't want to strip out the explanation the informative information. I don't know how to define TOC to keep the $(P) macro outside of the element which will enclose the TOCENTRY items. 2) Consider, for example, parsing arrays.dd (my comment can be easily applied to any other file). Unless I'm miscounting parentheses, $(H4) macros are not being used within $(H3) macros. Therefore, how do I get DDoc to parse the file so that it ends up with nested tags? for example: Dynamic Arrays Array Declarations Content 3) Consider, for example, this part from abi.dd: $(GRAMMAR $(I MangledName): $(B _D) $(I QualifiedName) $(I Type) $(B _D) $(I QualifiedName) $(B M) $(I Type) $(I QualifiedName): $(I SymbolName) $(I SymbolName) $(I QualifiedName) $(I SymbolName): $(I LName) $(I TemplateInstanceName) ) Say I want to style this using a descriptions list, the tag. That's easy enough, but now how do I tell DDoc to tag the $(I) macros using and tags? 4) Furthermore (still referring to the example above, because the issue applies to other areas), how do I tell DDoc that $(I)s within a $(GRAMMAR) macro are to be formatted using descriptions list syntax, but keep the other $(I) macros as regular elements? 5) The link-related macros appear, by and large, to use relative URLs. If I'm compiling only the DLang Spec into an ePUB, the standard, I believe, requires that the links be resolvable. That's easy enough if the relative URL in question points to another page in the spec. However, if the link points to another page on the website or a library document, which (for now) won't be in the ePUB, is the only way to identify and fix those links going to be by hand? These are the problems that I've run into thus far. I'm doing my best to appreciate the design and theory of DDoc, but maybe it's too much of a paradigm shift for me and I end up fighting the macros?
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 01:57:16 UTC, Andrei Alexandrescu wrote: This is a worthy goal. We manage to generate mobi files for the spec (and Phobos in a pull request), is the ebook format very different? Andrei Good evening, Professor, I'm still working through the ePUB standard, but, from what I can tell, the two are very similar. According to Wikipedia, ePub obsoletes Mobi (despite Amazon stubbornly enforcing Mobi (or some variant) for its Kindle). They both rely on similar standards, including OPF and NCX file formats, and ePUB uses XHTML and zip compression, thereby avoiding reinventing the wheel to the extent possible (although one would think that Docbook would make more sense for, you know, eBooks). Actually, in generating my first ePUB of the DLang spec, I simply used the same mobi input files, copied-and-pasted two boilerplate ePUB files and zipped the whole thing up. Aside from some egregious formatting on my Kobo, it works. Of course, I want to take the time to generate a proper ePUB 3 file and use as much of the standard as possible to advantage. Naturally, the goal is to extend whatever scripts I write to generate a DLang spec ePub to generate DDoc documentation eBooks, too, so I'm not entirely a one-issue person...
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 01:22:17 UTC, Borden wrote: 2) Adopting Latex's rule that a double line break means a new paragraph. This will effectively make the $(P) macros rampant in the DLang spec documentation unnecessary. Oops. I realised that this has already been done. OK, so I guess the question is why does the DLang spec need $(P) macros? How could it be rewritten without them and let dmd worry about them?
Re: DLang Spec rewrite (?)
I want to keep this discussion focussed on the DLang spec source code. If we want to debate the features of DDoc, we should do it in another thread. However, as not to appear full of cricism but short of ideas, I'm going to break my own rule and suggest, at least for the purposes of solving some of the issues I've run into with the DLang spec source, that integrating some wiki-markup into DDoc may help. For example: 1) Allowing sections to be defined using == Heading == or === Heading === instead of $(HEADING ) or variants. The advantage that Wiki syntax has over macro-syntax is that it automatically works out the section nesting (which is essential for building tables of contents in things such as, hint hint, eBooks) whereas macros can only do it if the subheadings are nested as arguments. 1a) Using ==Headings== and the existing /** */ code standards, DDoc could have a predefined $(TOC) macro which would auto-generate the TOC. /** */ would form the main headings and ==Heading== would be the subheadings, prettily nested when formatted. 2) Adopting Latex's rule that a double line break means a new paragraph. This will effectively make the $(P) macros rampant in the DLang spec documentation unnecessary. 3) Defining tables using the +---+ syntax. I know that this will be unpopular due to the existing /++ documentation code rules (and thus is open to alternatives). However, one must admire how simply Wiki markup has elegantly solved a problem that Latex and XML dosen't. 4) Using either * or - to indicate bullet points, similar to Wiki markup. Again, I know that it'll have to be coded as not to confuse the parser with /** */ and operands. 4a) Maybe #) to indicate ordered lists? (again, similar to Wiki markup) 5) Use the [[Link|Link name]] instead of $(Link) macros to cross-link. By default, Link would be a reference to some other DDoc and allow links to be handled automagically. How will this all help the DLang spec? Well, if the spec could be rewritten entirely in a /** */ block, with reasonable macro use, then couldn't it be parsed more readily into the necessary formats? Wouldn't it also make the source more readable and editable without all of the nested parentheses? Wouldn't the syntax be self-documenting? Anyway, just throwing stuff against the wall to see what sticks...
Re: DLang Spec rewrite (?)
On Sunday, 26 May 2013 at 00:28:05 UTC, Andrei Alexandrescu wrote: To generate several formats from one source, a macro system is needed. One interesting thing I figured about macro systems is they're all dirty - they can't be really considered "languages" because they intermix the programming part with the very output generated. So, what macro system would you use? (Actual question.) Look at m4 - it won't win any beauty contests, either, and it's enormously complicated. DDoc is simple for what it does, it has somehow hit a sweet spot. Good evening, Professor, I'm not arguing with the macro system or proposing a replacement. I think, for what it's designed to do, it works perfectly well and, you're right, is somewhat faster and cleaner than XML tags. (and this is from someone who's biased in favour of HTML tags) My contention is that, for the purposes of writing lengthy, non-code documentation like the DLang spec (I'm not referring to any other documentation or pages on the site), enclosing the entire exposition in macros has made the source too inflexible for me to work with without awkward workarounds or having to write my own parser. Again, the idea is to use the features of HTML5 and compile the DLang spec into an ePub document that I can read on my brand-new Kobo. Therefore, I'm not proposing a radical overhaul of DDoc or recommending that all known DDoc be recoded in HTML, as I mention in my other forum threads. Rather, I'm mentioning that, for my purposes, the DLang Spec source is too difficult for me to work with in its current state for my purposes and I'm offering to recode it in a language that will allow me to accomplish what I want to do; keep the DLang spec easy to read, write and maintain; and tread on as few toes as possible.
Re: DLang Spec rewrite (?)
On Saturday, 25 May 2013 at 23:28:46 UTC, Jonathan M Davis wrote: aside from you, I'm not aware of anyone complaining about it any time recently. Good evening, Jonathan, I'm not sure whether you mean that nobody's complained recently about the spec being in DDoc lately, because, as in my first link, I found that more people disliked the macros feature of DDoc (2 + me) than liked it (0). To answer your question, regarding concrete examples, these issues are in the context of wanting to translate the DLangSpec pages into HTML5 so that I can compile them into an ePUB document: 1) the DLangSpec files (pick any one) use SECTION# macros, where # is a number. In the DDoc conversion files to HTML, these SECTION# macros convert to tags and encase the contents within them. However, I'm not aware of any facility within DDoc, short of hand-writing a parser, to allow these SECTION# macros to be nested in order to take advantage of HTML5's tags. 2) The macros are not self-documenting. For example, consider $(LNAME2 pointers, Pointers) in arrays.dd. The easiest way, I know, to figure out what $(LNAME2) means is to read the posix.mak to see that arrays.dd gets pumped through ddoc.dd. Now, a search through doc.ddoc to find the declaration LINK2=$+ at last tells me that argument 1 is the path to the link and everything that follows that is the text to appear in the link. The point is that, as I struggle through modifying the existing .ddoc templates to compile to HTML5, I need to keep flipping back and forth between the source and the .ddoc to make sure that anything I'm redefining I'm doing correctly. 3) Again using LINK2, if I were to delete the LINK2= line from doc.ddoc and forget to readd it, my experience is that dmd -D will quietly drop instances of $(LINK2) without telling me. 4) Again using the same example, if LINK2 gets defined in multiple DDoc files, how do I know for certain which definition it calls when dmd runs against the files? 5) I find that a lot of the DLangSpec is written from an HTML point of view, so maybe it just needs rewriting to make the macros descriptive. For example, consider $(B dmd) and $(B -gc) on lines 881-882 of abi.dd. By default, these get converted into dmd and -gc Say I want commands (like dmd) to be bolded but I want command-line arguments not to be bolded. There's no way to write B= to single out some $(B)s and not others. Now, I know the knee-jerk response is "Yes, but HTML works the exact same way." That's true, but CSS *does* give you a bunch of selectors to cherry-pick, say, "only the tags of class X" or "the element with this id." Meaning that all I have to do is find the Bs I want to change and add a class="" without having to worry about updating any of the other Bs. Is there a simple way to do this in DDoc? These are just a few of the observations that I have. As I said in the other threads, my ePUB effort has ground to a halt because I find that I'm fighting to read the spec sources rather than figuring out how to produce clean and pretty HTML5 code that can get compiled into an ePUB.
Re: DLang Spec rewrite (?)
I hasten to add that I don't mean to criticise the original writers of the DLang Spec for writing it in DDoc macros. So far, I've found the documentation fairly easy to follow (as plain text) and so I don't want to lose any of that should the spec be rewritten. It's also possible (although, in my opinion, less preferable) to keep the spec written in DDoc macros but reformatted to allow for easier conversion to other formats...
DLang Spec rewrite (?)
Good afternoon, all, I would still like to compile the D Lang Spec into EPUB (and possibly other formats) but, as we discussed in these threads: http://forum.dlang.org/thread/bsbdpjyjubfxvmecw...@forum.dlang.org http://forum.dlang.org/thread/uzdngvjzexukbgkxd...@forum.dlang.org having the D Lang Specification written in DDoc macros is making it extremely difficult to work with. I ask, therefore, what opposition would there be to me rewriting the DLang Spec files into another format that will be easier to parse and compile for the website, PDF, Latex, eBook and other formats? If the answer is 'minimal', 'go ahead' or 'it's your funeral', then my follow-up question is 'what format would be the easiest to write, debug and maintain?' For greater clarity, I am NOT proposing to rewrite the DDoc-generated library documentation or any other pages outside of the spec. In the makefile, they are defined as the files covered in $(SPEC_ROOT). With regards,
Re: (X)HTML/XML in DDoc
On Thursday, 16 May 2013 at 21:36:03 UTC, Craig Dillabaugh wrote: Can you provide a reference for the claim that the next incarnation of TeX is going to be XML based? I did a quick search and couldn't turn up anything. Perhaps it was some sort of April fool's joke :o) You're right. I fell for it. I think I was referring to his announcement of iTeX, which any idiot should have noticed was a joke. I'm no idiot, I'm apparently far more gullible :-P
Re: (X)HTML/XML in DDoc
On Thursday, 16 May 2013 at 20:55:18 UTC, Adam D. Ruppe wrote: I think if you're using a lot of macros, ddoc starts to lose value because then it is harder to read in the code. The plainer the text, the better, IMO. I use ddoc macros only on the individual ($important word) here and there and sometimes little ($lists, of, simple, stuff). I think you're right, Adam. A standard 80-character line of code gets swallowed up pretty quickly in tags. Macros are certainly more concise but I've noticed that their main (ab)use is as HTML shorthand. I'm guilty of much the same: using macros to cut down on the amount of copy-and-pasting I have to do when I want to make a bunch of similar XML tags real quick. A corollary to this question is how much opposition I'd face if I were to rewrite the DLangSpec source files into a different markup. A corollary to the corollary is which standard would be easiest to write, read, understand and maintain?
Re: (X)HTML/XML in DDoc
I don't want to turn this thread into a DDoc-bashing rag, but another observation I've made is that, ironicaly, DDoc macros are not self documenting. If one types $(SOME_MACRO this, that, the other) it's not immediately obvious to what 'this,' 'that,' 'the other' refer without interpreting the macro definitions. Perhaps this was a design feature in order to transfer the formatting burden onto the .ddoc file, but I'm finding the exact opposite as I look through the DLangSpec. In contrast, the other is far more obvious in terms of what relationship the variables have to one another. Again, I'm not suggesting for a second that XML is the documenter's cure-all, but illustrating some of the things which make DDoc macros awkward to use. What are the general opinions on La(Tex) in terms of code documentation? I only have a superficial understanding of it but it seems to follow many of the principles that DDoc incorporates. It also has the advantage that it's not quite as obtrusive as XML when writing from scratch. Then again, I read on Wikipedia that Knuth's next incarnation of Tex is going to be XML-based. XML seems to be the unavoidable trend in file design.
Re: EPUB Documentation
On Thursday, 16 May 2013 at 19:23:37 UTC, Jonathan M Davis wrote: So, in this particular case, wrapping a C library would be a stop-gap solution at best. In other cases, it may be the best way to go, but parsing is one area where D stands out. Thank you for the insight, Jonathan, You have the advantage on me as you know far more about this and I know next to nothing. It's amusing, as a general observation, how object-oriented programming was supposed to make code reusable and yet it seems that these essential libraries constantly need rewriting in new languages because they're 'programmed wrong.' As I've mentioned in this thread http://forum.dlang.org/thread/bsbdpjyjubfxvmecw...@forum.dlang.org my EPUB conversion has hit something of a brick wall as I'm finding DDoc's macros to be very awkward to use and, in order to do what I need them to do, I'd have to rewrite much of the documentation! I'd be interested in feedback on that thread...
(X)HTML/XML in DDoc
Good evening, all, I imagine that this is a sensitive topic, so I'll do my best not to flame bait and I hope that readers will assume that I mean the best if I don't word things very diplomatically: I think that the DDoc spec does an excellent job in creating a common-sense, low-burdensome way to document source code in-line. I think its rules are largely clear and unambiguous. Unfortunately, I'm finding that the clarity breaks down once macros are introduced. What sticks out for me is the spec's aversion to XHTML, clearly stated in "D's goals for embedded documentation" (number 4) and under the bit for Embedded HTML, which discourages HTML in favour of macros. I want to understand why DDoc prefers macros to some XML-based markup. I've observed the following things about working with DDoc: 1) The DLangSpec is written almost entirely in macros. Most of the macros defined for the DLangSpec simply rewrite HTML tags into a macro. It seems that the macros copy the functionality of the HTML tags without adding any usefulness or clarity to them. 2) In fact, I'm finding that macros make the documentation less flexible to work with. For example, in trying to create a .ddoc to parse the DLangSpec files into HTML 5, I'm noticing that the DLangSpec sources define and use a lot of SECTION# macros, which simply turn the SECTION# heading into a and raw-dumps the second argument. Because of the way it's been set up, I have to find and redefine each of these SECTION# macros and, even when I do, I can't redefine the macros to use HTML5's nested tag system. 3) I do find the macros extremely helpful in mass-producing repetitive lines of HTML where only one or two attributes change in each line. Of course, since macros don't have any logic to them (such as looping, conditions, etc.), they can only shorten my workload so much. Maybe the problem is that the DLangSpec simply stretches the capabilities of DDoc far too much. I know that XML is verbose, redundant and often a nightmare to work with, but lots of tools are available to manage the tedium of XML and convert files to and from any format that you like - which seems to be what DDoc was designed to but can only accomplish after it's parsed into HTML! So these are my thoughts, and I invite feedback on them. I'm not really proposing anything - except for, maybe, changing the DLangSpec source into a more wieldy format...
Re: EPUB Documentation
On Wednesday, 15 May 2013 at 10:39:41 UTC, Russel Winder wrote: It might be worth noting here that SAX and DOM have not been distinguished so far in the thread. I would have thought SAX would be really easy in D. Indeed. My original question is specfic to XML document generation in D code, not necessarily reading (although any half-decent DOM library needs to be able to read files as well as write them). I'm not very familiar with SAX, but Wikipedia says that it's not standardised and so, if I understand correctly, there would be varying implementations of it. Although it might be nice to have D implementations, perhaps D might follow Python and wrap a rather good system so as to get something with full validation and XPath sooner rather than later? I think that this is a better approach to take. Surely Windows has an XML library and that would have to be done is write a single header file that would link to the XML library on the operating system? I know, probably far more difficult to implement than it sounds...
Re: EPUB Documentation
Thank you for the feedback, all. The XML parsing bit is a little important for me because, to generate the required boilerplate for an EPUB, one necessarily needs to manipulate XML. The approach I'm thinking for the EPUB is to write a 'generate EPUB from an OPF' program in D and shove it into the tools directory (hence my earlier question about what 'belongs' in there). Of course, I'd be happy to set straight to work in writing such a program (I have some XML experience using PHP and D could not possibly be any more awkward to use) but I don't want to be calling in libraries that are likely to be superceded before my patches get reviewed. As a hobbiest trained in accounting, I'm the last person anyone would want to work on a programming language. I suppose, though, that I'm pretty good with standards...
EPUB Documentation
Thank you very much, Walter, for your reply. I hope I've done it correctly. I may use this thread to post various questions about what I find in the repo as I stagger through it. As I consider how to approach this problem, here are some questions: 1) On the documentation page for std.xml, it states "Warning: This module is considered out-dated and not up to Phobos' current standards. It will remain until we have a suitable replacement, but be aware that it will not remain long term." Not just for the job I'm doing but for other XML manipulation I may do in the future, what is the recommended way to manipulate XML in D? Is there a task force (to the extent that such things exist in open source software) to address the shortcomings? 2) I'm trying to get a sense of the tools repo on GitHub. Am I correct in inferring that it consists of utilities to help the makefiles in the other repos? Is this project going to blossom into something else? With thanks,
Mobipocket to EPUB
Good evening, all, I appreciate the work of the people who've made the D Documentation, and I've wanted to download an eBook of the language specification to read offline. I see that the downloads page has the spec in Mobipocket format, but, according to wikipedia, it's been superceded by the more widely-adopted, and more open, EPUB format. To this end, I've downloaded the website source code from GitHub and I've been fiddling with posix.mak and its supporting files to compile the language spec into EPUB. I've been able (sorta) to get it to work, but I want to co-ordinate my effort with anyone else who's doing the migration. Further, after I've done my changes, what's the procedure to get my changes merged with GitHub? With thanks,
