Re: [Rd] NEWS.md support on CRAN
Sorry, require was the wrong word - suggest is better. I actually didn't realize when I first wrote a package that I should create a NEWS file because I was following the `package.skeleton()` route and that wasn't something that was generated automatically. I suspect other new package authors might be in the same place (i.e., just not aware that such a file is helpful for end users). Thomas J. Leeper http://www.thomasleeper.com On Fri, Jun 5, 2015 at 6:32 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 05/06/2015 8:16 AM, Thomas J. Leeper wrote: It's disappointing that many packages do not have a NEWS file. Perhaps CRAN should require NEWS or CHANGELOG, as long as the system is being reformed to potentially accommodate markdown anyway. I think we should all be careful about suggesting work for other people. This would be a huge amount of work, probably hundreds of hours in aggregate, for all those package authors and the CRAN maintainers. On the other hand, we can gently nudge package authors to better practices. When we talk in public about R packages, we should point out that we use a certain list of characteristics to judge their quality. My list includes: - Is it on CRAN? - Does it have an overview vignette? - Is it updated fairly regularly? - Do most of the help pages include examples? - Will it install on my OS without much trouble? Your list could include - Does it have a readable NEWS file? I do use packages which have negative answers to some of those questions, but they definitely influence my choice when I'm looking at a package for the first time. Duncan Murdoch -Thomas Thomas J. Leeper http://www.thomasleeper.com On Fri, Jun 5, 2015 at 12:00 PM, r-devel-requ...@r-project.org wrote: Date: Thu, 04 Jun 2015 12:39:34 -0400 From: Duncan Murdoch murdoch.dun...@gmail.com To: Mark van der Loo mark.vander...@gmail.com, kurt.hor...@wu.ac.at Cc: R Devel Mailing List r-devel@r-project.org, Dirk Eddelbuettel e...@debian.org, Imanuel Costigan i.costi...@me.com Subject: Re: [Rd] NEWS.md support on CRAN Message-ID: 55707f46.5010...@gmail.com Content-Type: text/plain; charset=utf-8; format=flowed On 04/06/2015 11:05 AM, Mark van der Loo wrote: FWIW (and a bit late in the discussion, I know), I for one do not care about having NEWS in md format at all. The solution the Yihui uses (linking to GH from NEWS.Rd) is really annoying for people with no direct Internet access. For example, I work at an institute that handles a lot of private data and most VM's with R on it have no direct internet access for that reason (of course internet is accessible but through an application running on a separate VM). Moreover, I as a user also do not care at all about links to GH #issues and which @user did what for each issue. These are details that are useful for people developing the package or for people who reported a bug. As a user I just want to read a short description like bugfix: function f crashed on input y, or function g is deprecated without having to first navigate to another website. The most important thing about the NEWS is that it is easy to find (so in a fixed place), and aimed at users, not developers. It should come with the software, so it is also available when GH is offline or replaced with something new (since hey, didn't we all have a sourceforge or google code account in our younger days?). In short, I think that added value of NEWS.md is fairly limited but it does increase the risk of dispersing the NEWS all over the web. I'd disagree a little bit: NEWS should be both for users and developers. The advantage I see to NEWS.md support is that it will encourage people who are more comfortable writing in Markdown to include the news in the package, rather than having a link to external news (as Yihui did), or no news at all (as many packages do). It's pretty easy to have plain text news, but links in the news are helpful. Links to bug reports are of most use to developers, but other links are useful to users. For example, though I don't think anyone is making use of this, it should be possible to have links from NEWS.Rd (or NEWS.md when it is fully supported) directly into R's help system. That would be especially useful for users. It would probably even work on isolated systems, as long as they don't block access to the local http server. Duncan Murdoch Best, Mark Op wo 3 jun. 2015 om 08:32 schreef Kurt Hornik kurt.hor...@wu.ac.at: Duncan Murdoch writes: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist
Re: [Rd] NEWS.md support on CRAN
On 05/06/2015 8:16 AM, Thomas J. Leeper wrote: It's disappointing that many packages do not have a NEWS file. Perhaps CRAN should require NEWS or CHANGELOG, as long as the system is being reformed to potentially accommodate markdown anyway. I think we should all be careful about suggesting work for other people. This would be a huge amount of work, probably hundreds of hours in aggregate, for all those package authors and the CRAN maintainers. On the other hand, we can gently nudge package authors to better practices. When we talk in public about R packages, we should point out that we use a certain list of characteristics to judge their quality. My list includes: - Is it on CRAN? - Does it have an overview vignette? - Is it updated fairly regularly? - Do most of the help pages include examples? - Will it install on my OS without much trouble? Your list could include - Does it have a readable NEWS file? I do use packages which have negative answers to some of those questions, but they definitely influence my choice when I'm looking at a package for the first time. Duncan Murdoch -Thomas Thomas J. Leeper http://www.thomasleeper.com On Fri, Jun 5, 2015 at 12:00 PM, r-devel-requ...@r-project.org wrote: Date: Thu, 04 Jun 2015 12:39:34 -0400 From: Duncan Murdoch murdoch.dun...@gmail.com To: Mark van der Loo mark.vander...@gmail.com, kurt.hor...@wu.ac.at Cc: R Devel Mailing List r-devel@r-project.org, Dirk Eddelbuettel e...@debian.org, Imanuel Costigan i.costi...@me.com Subject: Re: [Rd] NEWS.md support on CRAN Message-ID: 55707f46.5010...@gmail.com Content-Type: text/plain; charset=utf-8; format=flowed On 04/06/2015 11:05 AM, Mark van der Loo wrote: FWIW (and a bit late in the discussion, I know), I for one do not care about having NEWS in md format at all. The solution the Yihui uses (linking to GH from NEWS.Rd) is really annoying for people with no direct Internet access. For example, I work at an institute that handles a lot of private data and most VM's with R on it have no direct internet access for that reason (of course internet is accessible but through an application running on a separate VM). Moreover, I as a user also do not care at all about links to GH #issues and which @user did what for each issue. These are details that are useful for people developing the package or for people who reported a bug. As a user I just want to read a short description like bugfix: function f crashed on input y, or function g is deprecated without having to first navigate to another website. The most important thing about the NEWS is that it is easy to find (so in a fixed place), and aimed at users, not developers. It should come with the software, so it is also available when GH is offline or replaced with something new (since hey, didn't we all have a sourceforge or google code account in our younger days?). In short, I think that added value of NEWS.md is fairly limited but it does increase the risk of dispersing the NEWS all over the web. I'd disagree a little bit: NEWS should be both for users and developers. The advantage I see to NEWS.md support is that it will encourage people who are more comfortable writing in Markdown to include the news in the package, rather than having a link to external news (as Yihui did), or no news at all (as many packages do). It's pretty easy to have plain text news, but links in the news are helpful. Links to bug reports are of most use to developers, but other links are useful to users. For example, though I don't think anyone is making use of this, it should be possible to have links from NEWS.Rd (or NEWS.md when it is fully supported) directly into R's help system. That would be especially useful for users. It would probably even work on isolated systems, as long as they don't block access to the local http server. Duncan Murdoch Best, Mark Op wo 3 jun. 2015 om 08:32 schreef Kurt Hornik kurt.hor...@wu.ac.at: Duncan Murdoch writes: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc
Re: [Rd] NEWS.md support on CRAN
On 04/06/2015 17:00, Gavin Simpson wrote: ... 1. Markdown was designed to be as readable as possible in plain text format (i.e. the sources, not the rendered file). There's nothing stopping you reading and parsing a NEWS.md in your head trivially. ... That claim is made frequently, but it is not my experience. That you can filter out markup in your head does not mean others can -- it is very distracting to some visually impaired people (including when using a text-to-speech reader). -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford 1 South Parks Road, Oxford OX1 3TG, UK __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 04/06/2015 1:09 PM, Prof Brian Ripley wrote: On 04/06/2015 17:00, Gavin Simpson wrote: ... 1. Markdown was designed to be as readable as possible in plain text format (i.e. the sources, not the rendered file). There's nothing stopping you reading and parsing a NEWS.md in your head trivially. ... That claim is made frequently, but it is not my experience. That you can filter out markup in your head does not mean others can -- it is very distracting to some visually impaired people (including when using a text-to-speech reader). I agree. I think we shouldn't be displaying raw Markdown, except possibly as a temporary measure, while we get the parsing and rendering in place. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
Agreed, and There's nothing stopping you reading and parsing a NEWS.md in your head trivially. was very poorly phrased on my part. Just because I can filter this out doesn't mean it is easy for others to do so. Clearly anything with markup, even as light as markdown can be, is not as simple as plain text. G On 4 June 2015 at 11:09, Prof Brian Ripley rip...@stats.ox.ac.uk wrote: On 04/06/2015 17:00, Gavin Simpson wrote: ... 1. Markdown was designed to be as readable as possible in plain text format (i.e. the sources, not the rendered file). There's nothing stopping you reading and parsing a NEWS.md in your head trivially. ... That claim is made frequently, but it is not my experience. That you can filter out markup in your head does not mean others can -- it is very distracting to some visually impaired people (including when using a text-to-speech reader). -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford 1 South Parks Road, Oxford OX1 3TG, UK -- Gavin Simpson, PhD [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 4 June 2015 at 11:16, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 04/06/2015 1:09 PM, Prof Brian Ripley wrote: On 04/06/2015 17:00, Gavin Simpson wrote: ... 1. Markdown was designed to be as readable as possible in plain text format (i.e. the sources, not the rendered file). There's nothing stopping you reading and parsing a NEWS.md in your head trivially. ... That claim is made frequently, but it is not my experience. That you can filter out markup in your head does not mean others can -- it is very distracting to some visually impaired people (including when using a text-to-speech reader). I agree. I think we shouldn't be displaying raw Markdown, except possibly as a temporary measure, while we get the parsing and rendering in place. Just to be clear; I wasn't suggesting that R should be displaying raw Markdown. What I was getting at was even if your local setup stops you seeing a rendered version of the file (for whatever reason) reading the Markdown source is a lot easier than reading a similar file marked up using Rd. Yet for those with less restrictive setups Markdown maintains many of the benefits that the Rd markup allows. I just failed miserably, and insensitively(!), to convey that point. G Duncan Murdoch -- Gavin Simpson, PhD [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
Mark, With respect I think you're conflating two different things here. I completely agree with you that the workflow Yihui uses where the NEWS file doesn't actually have the NEWS in it in a form that R can understand is quite bad for users. It also makes the NEWS difficult to compute on, which has caused me problems. That is entirely separate from what formats R allows and understands the NEWS within the package to be declared in, though. I think most people are talking about supporting the shipping of NEWS in markdown *within the package*, largely because people just don't like writing Rd files. It's painful, so many people don't do it. It's easy to say they should do it anyway, and you wouldn't be wrong, but unfortunately that isn't likely to sway many people who are busy and have other things they want to do with their time. Allowing package authors to write a NEWS.md file - and, again, ship it with the package - eases that pain. With the type of support Duncan and others are talking about, the end user (you) wouldn't even know if the package had a NEWS.md or NEWS.Rd file, the result would be the same from within R. Best, ~G On Thu, Jun 4, 2015 at 8:05 AM, Mark van der Loo mark.vander...@gmail.com wrote: FWIW (and a bit late in the discussion, I know), I for one do not care about having NEWS in md format at all. The solution the Yihui uses (linking to GH from NEWS.Rd) is really annoying for people with no direct Internet access. For example, I work at an institute that handles a lot of private data and most VM's with R on it have no direct internet access for that reason (of course internet is accessible but through an application running on a separate VM). Moreover, I as a user also do not care at all about links to GH #issues and which @user did what for each issue. These are details that are useful for people developing the package or for people who reported a bug. As a user I just want to read a short description like bugfix: function f crashed on input y, or function g is deprecated without having to first navigate to another website. The most important thing about the NEWS is that it is easy to find (so in a fixed place), and aimed at users, not developers. It should come with the software, so it is also available when GH is offline or replaced with something new (since hey, didn't we all have a sourceforge or google code account in our younger days?). In short, I think that added value of NEWS.md is fairly limited but it does increase the risk of dispersing the NEWS all over the web. Best, Mark Op wo 3 jun. 2015 om 08:32 schreef Kurt Hornik kurt.hor...@wu.ac.at: Duncan Murdoch writes: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc options that emulate Github, it would probably make sense for CRAN to use those. Sure. We currently have pandoc 1.12.4.2 Compiled with texmath 0.6.6.1, highlighting-kate 0.5.8.5. which we use with --email-obfuscation=references. Best -k In the longer term, the plan is to include our own parser and renderer. At that point this would be easy. Duncan Murdoch Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page.
Re: [Rd] NEWS.md support on CRAN
Duncan Murdoch writes: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc options that emulate Github, it would probably make sense for CRAN to use those. Sure. We currently have pandoc 1.12.4.2 Compiled with texmath 0.6.6.1, highlighting-kate 0.5.8.5. which we use with --email-obfuscation=references. Best -k In the longer term, the plan is to include our own parser and renderer. At that point this would be easy. Duncan Murdoch Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page. (Would | need the CRAN incoming checks to be taught about inst/NEWS.md.) | | However, we cannot use such files for utils::news() because we do not | (yet?) know how to reliably parse such files and extract the news items | (and hence cannot really compute on the news information). | | Btw, currently only one package on CRAN has inst/NEWS.md (another one | has NEWS.md at top level). | | Best | -k | | Markdown is allowed for vignettes (because the package author processes | those), so I'd suggest putting your news into a vignette instead of a | news file. Put in a token news file that points to the vignette so | users can find it. | | Duncan Murdoch | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | [[alternative HTML version deleted]] | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | Peter Dalgaard, Professor, | Center for Statistics, Copenhagen Business School | Solbjerg Plads 3, 2000 Frederiksberg, Denmark | Phone: (+45)38153501 | Email: pd@cbs.dk Priv: pda...@gmail.com | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 02/06/2015 11:42 AM, Dirk Eddelbuettel wrote: On 2 June 2015 at 11:25, Duncan Murdoch wrote: | On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: | Hi Kurt, | | On 1 June 2015 at 14:02, Kurt Hornik wrote: | | peter dalgaard writes: | | | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | | | | in the development version. Not true of released versions. | | | | Now also in r-patched. | | Nice. | | Now, is there a way for package authors to preview how a .md would be | rendered? I wrote mine with GitHub in mind, and they render fine. I looked a | recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y | parts wrong --- and looks unprofessional. | | I would like to avoid that. How can I? | | In the short term, you should probably try to run pandoc with the same | version and options as CRAN. Kurt, can you say what these are? If you | (Dirk) know pandoc options that emulate Github, it would probably make | sense for CRAN to use those. | | In the longer term, the plan is to include our own parser and renderer. | At that point this would be easy. Thumbs up -- the 'rmarkdown' package by JJ et al basically already does just this, and I have now authored a few (beamer) presentations that way. And quite like it. It is all fully scriptable -- and of course available interactive in whatever R shell you prefer -- and only requires pandoc besides what is on CRAN. Maybe medium-term Jeroen's commonmark can replace pandoc and then we'd fully based on CRAN packages. Several of us have been talking about this, and have more or less decided against pandoc for the within-R version. It is too hard to build, or to include with R, on weird platforms. Something like commonmark is more likely. We'd also like to separate the parse and render steps. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Tue, Jun 2, 2015 at 8:42 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: Several of us have been talking about this, and have more or less decided against pandoc for the within-R version. It is too hard to build, or to include with R, on weird platforms. Something like commonmark is more likely. We'd also like to separate the parse and render steps. The commonmark::markdown_xml function gives the parse tree in xml format. From there we could use something like xpath to query the content for individual news items and such. I think this is a more practical way of exposing the syntax tree than deeply nested lists or an external pointer to the internal C handle. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 02/06/2015 3:40 PM, Jeroen Ooms wrote: On Tue, Jun 2, 2015 at 8:42 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: Several of us have been talking about this, and have more or less decided against pandoc for the within-R version. It is too hard to build, or to include with R, on weird platforms. Something like commonmark is more likely. We'd also like to separate the parse and render steps. The commonmark::markdown_xml function gives the parse tree in xml format. From there we could use something like xpath to query the content for individual news items and such. I think this is a more practical way of exposing the syntax tree than deeply nested lists or an external pointer to the internal C handle. I'd rather avoid adding a dependence on the XML package; it's not a base or recommended package, so most people don't have it. Base R doesn't have any XML processing built in. One nice thing about the commonmark library is that it is self-contained, so we can add it to R without pulling in a lot of other stuff. Using it to translate Markdown to XML loses that. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Tue, Jun 2, 2015 at 12:40 PM, Jeroen Ooms jeroeno...@gmail.com wrote: On Tue, Jun 2, 2015 at 8:42 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: Several of us have been talking about this, and have more or less decided against pandoc for the within-R version. It is too hard to build, or to include with R, on weird platforms. Something like commonmark is more likely. We'd also like to separate the parse and render steps. The commonmark::markdown_xml function gives the parse tree in xml format. From there we could use something like xpath to query the content for individual news items and such. I think this is a more practical way of exposing the syntax tree than deeply nested lists or an external pointer to the internal C handle. I don't think it should matter what the internal representation is here. So long as there is proper tooling (query methods, [ and [[, names, etc) around whatever class is returned, the user won't ever even know. XPath is certainly a good way to implement those things, but I don't know if we'd want it to be the actual abstraction users/developers use to interact with the tree. Thinking about the right way to consume ASTs represented by R objects (and how those objects should be structured/implemented) could have applications well beyond markdown as well, some of which may not have a canonical/direct XML representation. ~G __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Gabriel Becker, PhD Computational Biologist Bioinformatics and Computational Biology Genentech, Inc. [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page. (Would | need the CRAN incoming checks to be taught about inst/NEWS.md.) | | However, we cannot use such files for utils::news() because we do not | (yet?) know how to reliably parse such files and extract the news items | (and hence cannot really compute on the news information). | | Btw, currently only one package on CRAN has inst/NEWS.md (another one | has NEWS.md at top level). | | Best | -k | | Markdown is allowed for vignettes (because the package author processes | those), so I'd suggest putting your news into a vignette instead of a | news file. Put in a token news file that points to the vignette so | users can find it. | | Duncan Murdoch | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | [[alternative HTML version deleted]] | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | Peter Dalgaard, Professor, | Center for Statistics, Copenhagen Business School | Solbjerg Plads 3, 2000 Frederiksberg, Denmark | Phone: (+45)38153501 | Email: pd@cbs.dk Priv: pda...@gmail.com | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 2 June 2015 at 11:25, Duncan Murdoch wrote: | On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: | Hi Kurt, | | On 1 June 2015 at 14:02, Kurt Hornik wrote: | | peter dalgaard writes: | | | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | | | | in the development version. Not true of released versions. | | | | Now also in r-patched. | | Nice. | | Now, is there a way for package authors to preview how a .md would be | rendered? I wrote mine with GitHub in mind, and they render fine. I looked a | recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y | parts wrong --- and looks unprofessional. | | I would like to avoid that. How can I? | | In the short term, you should probably try to run pandoc with the same | version and options as CRAN. Kurt, can you say what these are? If you | (Dirk) know pandoc options that emulate Github, it would probably make | sense for CRAN to use those. | | In the longer term, the plan is to include our own parser and renderer. | At that point this would be easy. Thumbs up -- the 'rmarkdown' package by JJ et al basically already does just this, and I have now authored a few (beamer) presentations that way. And quite like it. It is all fully scriptable -- and of course available interactive in whatever R shell you prefer -- and only requires pandoc besides what is on CRAN. Maybe medium-term Jeroen's commonmark can replace pandoc and then we'd fully based on CRAN packages. Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
I realise this is not directly related to the topic but if changes are afoot would it be possible to make the use of any sort of news or changelog more prominent in the documentation? I maintain one of the CRAN task views and when someone updates his/her package it is much, much easier for me to see what has happened and if I need to update the text if they supply a news or changelog. On 02/06/2015 16:25, Duncan Murdoch wrote: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc options that emulate Github, it would probably make sense for CRAN to use those. In the longer term, the plan is to include our own parser and renderer. At that point this would be easy. Duncan Murdoch Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page. (Would | need the CRAN incoming checks to be taught about inst/NEWS.md.) | | However, we cannot use such files for utils::news() because we do not | (yet?) know how to reliably parse such files and extract the news items | (and hence cannot really compute on the news information). | | Btw, currently only one package on CRAN has inst/NEWS.md (another one | has NEWS.md at top level). | | Best | -k | | Markdown is allowed for vignettes (because the package author processes | those), so I'd suggest putting your news into a vignette instead of a | news file. Put in a token news file that points to the vignette so | users can find it. | | Duncan Murdoch | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | [[alternative HTML version deleted]] | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | Peter Dalgaard, Professor, | Center for Statistics, Copenhagen Business School | Solbjerg Plads 3, 2000 Frederiksberg, Denmark | Phone: (+45)38153501 | Email: pd@cbs.dk Priv: pda...@gmail.com | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Michael http://www.dewey.myzen.co.uk/home.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc options that emulate Github, it would probably make sense for CRAN to use those. In the longer term, the plan is to include our own parser and renderer. At that point this would be easy. Duncan Murdoch Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page. (Would | need the CRAN incoming checks to be taught about inst/NEWS.md.) | | However, we cannot use such files for utils::news() because we do not | (yet?) know how to reliably parse such files and extract the news items | (and hence cannot really compute on the news information). | | Btw, currently only one package on CRAN has inst/NEWS.md (another one | has NEWS.md at top level). | | Best | -k | | Markdown is allowed for vignettes (because the package author processes | those), so I'd suggest putting your news into a vignette instead of a | news file. Put in a token news file that points to the vignette so | users can find it. | | Duncan Murdoch | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | [[alternative HTML version deleted]] | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | Peter Dalgaard, Professor, | Center for Statistics, Copenhagen Business School | Solbjerg Plads 3, 2000 Frederiksberg, Denmark | Phone: (+45)38153501 | Email: pd@cbs.dk Priv: pda...@gmail.com | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
This is supported by the NEWS.Rd mechanism, but many authors don't use it. I agree; it would be wonderful if everyone used it and I think the main (potential) advantage of this thread is to make it used (both by developers and users) more broadly. Kasper On Tue, Jun 2, 2015 at 11:42 AM, Michael Dewey li...@dewey.myzen.co.uk wrote: I realise this is not directly related to the topic but if changes are afoot would it be possible to make the use of any sort of news or changelog more prominent in the documentation? I maintain one of the CRAN task views and when someone updates his/her package it is much, much easier for me to see what has happened and if I need to update the text if they supply a news or changelog. On 02/06/2015 16:25, Duncan Murdoch wrote: On 02/06/2015 11:05 AM, Dirk Eddelbuettel wrote: Hi Kurt, On 1 June 2015 at 14:02, Kurt Hornik wrote: | peter dalgaard writes: | | On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: | | So I assume this commit means NEWS.md is now no longer on blacklist? | | | in the development version. Not true of released versions. | | Now also in r-patched. Nice. Now, is there a way for package authors to preview how a .md would be rendered? I wrote mine with GitHub in mind, and they render fine. I looked a recently-uploaded README.md of mine on CRAN, and it got some of the pandoc-y parts wrong --- and looks unprofessional. I would like to avoid that. How can I? In the short term, you should probably try to run pandoc with the same version and options as CRAN. Kurt, can you say what these are? If you (Dirk) know pandoc options that emulate Github, it would probably make sense for CRAN to use those. In the longer term, the plan is to include our own parser and renderer. At that point this would be easy. Duncan Murdoch Dirk | -k | | -pd | | | https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a | | Sent from my iPad | | On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: | | Duncan Murdoch writes: | | On 22/05/2015 8:49 PM, Imanuel Costigan wrote: | Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. | | Not as far as I know. There have been discussions about increasing the | support of Markdown, but so far the conclusion has been that it's too | hard to do -- the support is not stable enough on all the platforms | where R runs. | | There are actually two issues here. | | For CRAN, we could in principle take inst/NEWS.md files, convert these | to HTML using pandoc, and use the HTML for the package web page. (Would | need the CRAN incoming checks to be taught about inst/NEWS.md.) | | However, we cannot use such files for utils::news() because we do not | (yet?) know how to reliably parse such files and extract the news items | (and hence cannot really compute on the news information). | | Btw, currently only one package on CRAN has inst/NEWS.md (another one | has NEWS.md at top level). | | Best | -k | | Markdown is allowed for vignettes (because the package author processes | those), so I'd suggest putting your news into a vignette instead of a | news file. Put in a token news file that points to the vignette so | users can find it. | | Duncan Murdoch | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | [[alternative HTML version deleted]] | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | -- | Peter Dalgaard, Professor, | Center for Statistics, Copenhagen Business School | Solbjerg Plads 3, 2000 Frederiksberg, Denmark | Phone: (+45)38153501 | Email: pd@cbs.dk Priv: pda...@gmail.com | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel | | __ | R-devel@r-project.org mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Michael http://www.dewey.myzen.co.uk/home.html __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
peter dalgaard writes: On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: So I assume this commit means NEWS.md is now no longer on blacklist? in the development version. Not true of released versions. Now also in r-patched. -k -pd https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a Sent from my iPad On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). Btw, currently only one package on CRAN has inst/NEWS.md (another one has NEWS.md at top level). Best -k Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 30 May 2015, at 01:20 , Imanuel Costigan i.costi...@me.com wrote: So I assume this commit means NEWS.md is now no longer on blacklist? in the development version. Not true of released versions. -pd https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a Sent from my iPad On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). Btw, currently only one package on CRAN has inst/NEWS.md (another one has NEWS.md at top level). Best -k Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Peter Dalgaard, Professor, Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
So I assume this commit means NEWS.md is now no longer on blacklist? https://github.com/wch/r-source/commit/9ffe87264a1cd59a31a829f72d57af0f1bfa327a Sent from my iPad On 23 May 2015, at 6:05 pm, Kurt Hornik kurt.hor...@wu.ac.at wrote: Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). Btw, currently only one package on CRAN has inst/NEWS.md (another one has NEWS.md at top level). Best -k Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. In answer to your original question: plans are now being formed. It seems as though full support should be feasible, but someone needs to volunteer to do it and maintain it. By full support, I mean support that's equal to the support for NEWS.Rd on CRAN and in R. It's possible we'll get weaker support sooner, e.g. display of the Markdown source in R and pandoc output on CRAN; that remains to be determined. No predicted time line for any of this. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 10:26 PM, Imanuel Costigan wrote: On 24 May 2015, at 12:07 pm, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:15 PM, Imanuel Costigan wrote: While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to see” this file in the Help pane in RStudio That isn't really any simpler. RStudio is just displaying HTML whenever it shows you anything in the Help pane. Ok yes, point taken. My post was more in relation to a short-term “fix” of being able to save the NEWS.md file in the package’s top level directory. Users could still be able to read it as a plain text file in their R session (esp. if they don’t have web access) AND be able to see the pretty marked up version on Github if they wished. At the moment, it isn’t possible to make this work without triggering CRAN errors (by storing it in the top-level) or losing the NEWS.md file from top level directory of the package (by saving to inst/) and making it less conventional / accessible on Github. Ideally, one should be able to get the best of both: save this in top-level directory and when necessary, just present it as a text file (at least until such time as Markdown is officially supported). or being about to run something like show_news(“packagename”). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? I don't understand your question. What issues did I mention? Or are you talking about Kurt's post, who first mentioned news()? And what structure are you talking about? Yes I was referring to Kurt’s comments. As I understand it, the short-term “fix” I outlined above wouldn’t work because news() expects a certain structure and can’t extract the elements that it expects from Markdown files yet. What I am asking is why it isn’t possible / desirable for news() to simply print to the console the contents of `system.file(“NEWS.md”, package = “packagename”)`? For example, `news(package = “devtools”)` returns nothing because it uses “NEWS.md”. Short term fixes are generally a bad way to design software. We should do this right if we do it at all. It might require people using Github to change the way they do things, if that turns out to make more sense than accommodating them. Duncan Murdoch Duncan Murdoch On 24 May 2015, at 10:51 am, Baptiste Auguie baptiste.aug...@gmail.com wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in
Re: [Rd] NEWS.md support on CRAN
John MacFarlane, the author of Pandoc, has been working on a project ( http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 24 May 2015 at 14:01, Duncan Murdoch murdoch.dun...@gmail.com wrote: It sounds as though there are at least two possibilities for parsers that could be included in R: Sundown and commonmark. Sundown is in fact deprecated, https://github.com/vmg/sundown/commit/37728fb2d7137ff7c37d0a474cb827a8d6d846d8 in favor of commonmark which will likely become the standard tool for StackExchange, github, Reddit, among others. The markdown package has been somewhat superseded by rmarkdown, based on Pandoc. However commonmark would be easier to include (no dependency on Haskell), much faster, and more memory efficient. The good folks at Rstudio will probably have considered this option, and may even have started working on R bindings. Commonmark seems to use an intermediate Abstract Syntax Tree, which could be also useful for custom processing. Regards, baptiste [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 24/05/2015 7:20 AM, Duncan Murdoch wrote: On 23/05/2015 10:26 PM, Imanuel Costigan wrote: On 24 May 2015, at 12:07 pm, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:15 PM, Imanuel Costigan wrote: While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to see” this file in the Help pane in RStudio That isn't really any simpler. RStudio is just displaying HTML whenever it shows you anything in the Help pane. Ok yes, point taken. My post was more in relation to a short-term “fix” of being able to save the NEWS.md file in the package’s top level directory. Users could still be able to read it as a plain text file in their R session (esp. if they don’t have web access) AND be able to see the pretty marked up version on Github if they wished. At the moment, it isn’t possible to make this work without triggering CRAN errors (by storing it in the top-level) or losing the NEWS.md file from top level directory of the package (by saving to inst/) and making it less conventional / accessible on Github. Ideally, one should be able to get the best of both: save this in top-level directory and when necessary, just present it as a text file (at least until such time as Markdown is officially supported). or being about to run something like show_news(“packagename”). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? I don't understand your question. What issues did I mention? Or are you talking about Kurt's post, who first mentioned news()? And what structure are you talking about? Yes I was referring to Kurt’s comments. As I understand it, the short-term “fix” I outlined above wouldn’t work because news() expects a certain structure and can’t extract the elements that it expects from Markdown files yet. What I am asking is why it isn’t possible / desirable for news() to simply print to the console the contents of `system.file(“NEWS.md”, package = “packagename”)`? For example, `news(package = “devtools”)` returns nothing because it uses “NEWS.md”. Short term fixes are generally a bad way to design software. We should do this right if we do it at all. It might require people using Github to change the way they do things, if that turns out to make more sense than accommodating them. I imagine GitHub users could have both NEWS.md and NEWS, with one being a symlink to the other, and .Rbuildignore set to ignore NEWS.md. Why not try it, and post instructions for other Github users? news() won't be able to understand the headings, but it should display the file as a bunch of text. Duncan Murdoch Duncan Murdoch Duncan Murdoch On 24 May 2015, at 10:51 am, Baptiste Auguie baptiste.aug...@gmail.com wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable
Re: [Rd] NEWS.md support on CRAN
I like the idea of supporting a small, strict subset of Markdown that can be used to translate from NEWS.md to NEWS. Following from Yihui's example, it would be pretty easy to write a parser in R for such a format (and I'd be willing to try implementing one, if that would be of interest). Kevin On Sun, May 24, 2015 at 10:20 AM, Yihui Xie x...@yihui.name wrote: That is more or less what I had been doing for a long time (having both NEWS.md and NEWS), but decided not to do it any more last year. In fact, you can easily convert NEWS.md to a NEWS file that R's news() can understand, e.g. https://github.com/yihui/knitr/blob/947ad5fc94/Makefile#L8-L10 (if your NEWS.md is like this https://raw.githubusercontent.com/yihui/knitr/947ad5fc94/NEWS.md) I stopped doing this because as I said, I found Github release notes much more pleasant to read (https://github.com/yihui/knitr/releases), and I do not care much about the possibility that some users do not have internet connections when reading the NEWS (it is a legitimate concern, though). IMHO, it is totally worth it if we are talking about official support of Markdown in R's documentation system (.Rd files), and it is probably not worth the time and effort if we only want to support NEWS.md in particular. That is just a tiny problem compared to the effort of porting CommonMark or whatever Markdown rendering engines into R. Regards, Yihui On Sun, May 24, 2015 at 7:58 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: I imagine GitHub users could have both NEWS.md and NEWS, with one being a symlink to the other, and .Rbuildignore set to ignore NEWS.md. Why not try it, and post instructions for other Github users? news() won't be able to understand the headings, but it should display the file as a bunch of text. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
That is more or less what I had been doing for a long time (having both NEWS.md and NEWS), but decided not to do it any more last year. In fact, you can easily convert NEWS.md to a NEWS file that R's news() can understand, e.g. https://github.com/yihui/knitr/blob/947ad5fc94/Makefile#L8-L10 (if your NEWS.md is like this https://raw.githubusercontent.com/yihui/knitr/947ad5fc94/NEWS.md) I stopped doing this because as I said, I found Github release notes much more pleasant to read (https://github.com/yihui/knitr/releases), and I do not care much about the possibility that some users do not have internet connections when reading the NEWS (it is a legitimate concern, though). IMHO, it is totally worth it if we are talking about official support of Markdown in R's documentation system (.Rd files), and it is probably not worth the time and effort if we only want to support NEWS.md in particular. That is just a tiny problem compared to the effort of porting CommonMark or whatever Markdown rendering engines into R. Regards, Yihui On Sun, May 24, 2015 at 7:58 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: I imagine GitHub users could have both NEWS.md and NEWS, with one being a symlink to the other, and .Rbuildignore set to ignore NEWS.md. Why not try it, and post instructions for other Github users? news() won't be able to understand the headings, but it should display the file as a bunch of text. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Sun, May 24, 2015 at 4:01 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: The markdown package does some of what R would need to make use of Sundown, but not all: we really do need an R object representation of the parse tree, for functions like news() to work with in the short term, and the help system in the longer term. John MacFarlane's reference implementation does provide the markdown parsing tree, although by default it is only exposed in xml form. A quick example: devtools::install_github(jeroenooms/commonmark) help(commonmark) From the xml tree you can easily extract news items using xpath, or we could hack the libcmark library a bit to get the parse tree as an R list. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 11:31 AM, Yihui Xie wrote: I agree. It is not worth all the trouble just to save the bit of hassle to go to the package's Github site. I'm sorry, but in my opinion this is not negotiable. R needs to run on systems without Internet connections. It's not just a bit of hassle. R is used in places where not everyone has good Internet all the time. In fact, the release notes on Github are more meaningful than a plain text NEWS.md or even a converted NEWS.html from Pandoc, e.g. you can include bug report numbers and attribute to users by @username (they all have hyperlinks attached on them, so it is easy to see more details of bugs/features if one really cares). Personally, I feel it is very worthwhile going to Github and reading the release notes there. I'd be unhappy with converting NEWS.md to NEWS.html by Pandoc. I know not all people use Github, but I feel if a package author has a NEWS.md, chances are this package is on Github. Yes, NEWS.md would allow richer content than NEWS does. That's true if it is hosted on Github or not. (So does NEWS.Rd, but it is so inconvenient to write, that I think a local copy of NEWS.md would be a better solution.) Duncan Murdoch Re Kurt's analysis of NEWS.md on CRAN, I guess that is because R CMD check will warn against NEWS.md at the top level. I know a lot of packages on Github have the NEWS.md file, and it has been removed from the tarball to make R CMD check happy. Regards, Yihui -- Yihui Xie xieyi...@gmail.com Web: http://yihui.name On Sat, May 23, 2015 at 8:25 AM, Gábor Csárdi csardi.ga...@gmail.com wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Sat, May 23, 2015 at 8:00 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: [...] I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. markdown-it is already in an R package, so all you need is devtools::install_github(jeroenooms/markdownit) to try it. Actually an even better alternative is 'sundown' ( https://github.com/vmg/sundown), which already has R bindings, in the 'markdown' package. I should have suggested this in the first place, sorry for missing it. [...] Gabor [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to see” this file in the Help pane in RStudio or being about to run something like show_news(“packagename”). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? On 24 May 2015, at 10:51 am, Baptiste Auguie baptiste.aug...@gmail.com wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 8:51 PM, Baptiste Auguie wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark It sounds as though there are at least two possibilities for parsers that could be included in R: Sundown and commonmark. The markdown package does some of what R would need to make use of Sundown, but not all: we really do need an R object representation of the parse tree, for functions like news() to work with in the short term, and the help system in the longer term. To allow NEWS.md to be used, we'd also need someone to work out the conventions for what's allowed (probably very similar to the conventions for NEWS or NEWS.Rd), and write code to work with those files. Does anyone want to work on this? Duncan Murdoch Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __ R-devel@r-project.org mailto:R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
I’d suggest going with CommonMark if this will be the basis of broader support for Markdown as it is well defined whereas GruberMarkdown is defined by broad conventions with specifics determined by parsers. On 24 May 2015, at 12:01 pm, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 8:51 PM, Baptiste Auguie wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark It sounds as though there are at least two possibilities for parsers that could be included in R: Sundown and commonmark. The markdown package does some of what R would need to make use of Sundown, but not all: we really do need an R object representation of the parse tree, for functions like news() to work with in the short term, and the help system in the longer term. To allow NEWS.md to be used, we'd also need someone to work out the conventions for what's allowed (probably very similar to the conventions for NEWS or NEWS.Rd), and write code to work with those files. Does anyone want to work on this? Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 9:15 PM, Imanuel Costigan wrote: While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to see” this file in the Help pane in RStudio That isn't really any simpler. RStudio is just displaying HTML whenever it shows you anything in the Help pane. or being about to run something like show_news(“packagename”). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? I don't understand your question. What issues did I mention? Or are you talking about Kurt's post, who first mentioned news()? And what structure are you talking about? Duncan Murdoch On 24 May 2015, at 10:51 am, Baptiste Auguie baptiste.aug...@gmail.com wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 24 May 2015, at 12:07 pm, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:15 PM, Imanuel Costigan wrote: While a parsed HTML version of the NEWS.md file would be nice, I would like something much simpler: being able to see” this file in the Help pane in RStudio That isn't really any simpler. RStudio is just displaying HTML whenever it shows you anything in the Help pane. Ok yes, point taken. My post was more in relation to a short-term “fix” of being able to save the NEWS.md file in the package’s top level directory. Users could still be able to read it as a plain text file in their R session (esp. if they don’t have web access) AND be able to see the pretty marked up version on Github if they wished. At the moment, it isn’t possible to make this work without triggering CRAN errors (by storing it in the top-level) or losing the NEWS.md file from top level directory of the package (by saving to inst/) and making it less conventional / accessible on Github. Ideally, one should be able to get the best of both: save this in top-level directory and when necessary, just present it as a text file (at least until such time as Markdown is officially supported). or being about to run something like show_news(“packagename”). Duncan mentioned issues with the news() function being able to process metadata represented in the Md file. What is the motivation of this structure? I don't understand your question. What issues did I mention? Or are you talking about Kurt's post, who first mentioned news()? And what structure are you talking about? Yes I was referring to Kurt’s comments. As I understand it, the short-term “fix” I outlined above wouldn’t work because news() expects a certain structure and can’t extract the elements that it expects from Markdown files yet. What I am asking is why it isn’t possible / desirable for news() to simply print to the console the contents of `system.file(“NEWS.md”, package = “packagename”)`? For example, `news(package = “devtools”)` returns nothing because it uses “NEWS.md”. Duncan Murdoch On 24 May 2015, at 10:51 am, Baptiste Auguie baptiste.aug...@gmail.com wrote: John MacFarlane, the author of Pandoc, has been working on a project (http://commonmark.org/) to define a standard reference for Markdown*. There are already two reference implementations, one in javascript, the other in C: https://github.com/jgm/cmark Regards, baptiste * There was some initial controversy with the original author of markdown, but in the long term it's probably one of the more reliable sources to follow. On 24 May 2015 at 12:00, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 23/05/2015 9:25 AM, Gábor Csárdi wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com mailto:murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. I wouldn't want R builds to depend on GitHub, so this sounds more interesting. I took a look at that website, and it looks problematic to me: the parser appears to be written in Javascript, and the install instructions (using npm and bower, whatever those are) depend on some unstated prerequisites. In principle there's no reason not to allow R builds to depend on these things, but adding a dependency like that implies so much testing that I can't imagine anyone who could do it would want to. It's likely that a suitable parser could be written in some combination of C and R -- Markdown is not a complicated language. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. But we do need something, and it needs to be essentially universally available, or small enough to include in the R sources. I think R should eventually support Markdown as an acceptable language for documentation (including NEWS.md, and also help files for functions), but I think the effort required to do it now is too much. Duncan Murdoch Gabor __
Re: [Rd] NEWS.md support on CRAN
Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). Btw, currently only one package on CRAN has inst/NEWS.md (another one has NEWS.md at top level). Best -k Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 4:05 AM, Kurt Hornik wrote: Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). It would be quite easy to modify the news() parser to parse a suitably described Markdown format. The main change from the current text parser would be to expect a prefix on the line introducing each version or category. I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. I could add it to Rtools on Windows, but I think platforms that normally install tools from source will have more trouble, because it has an unusual prerequisite (Haskell). We could fall back to displaying the NEWS.md file without processing, but that makes lots of code more complicated. Btw, currently only one package on CRAN has inst/NEWS.md (another one has NEWS.md at top level). I'd guess if someone solved the problems mentioned above, it would become more popular. Many people would convert their text NEWS to NEWS.md if it would display better; that's a lot easier than converting to NEWS.Rd. Duncan Murdoch Best -k Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23/05/2015 13:14, Duncan Murdoch wrote: On 23/05/2015 4:05 AM, Kurt Hornik wrote: Duncan Murdoch writes: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. There are actually two issues here. For CRAN, we could in principle take inst/NEWS.md files, convert these to HTML using pandoc, and use the HTML for the package web page. (Would need the CRAN incoming checks to be taught about inst/NEWS.md.) However, we cannot use such files for utils::news() because we do not (yet?) know how to reliably parse such files and extract the news items (and hence cannot really compute on the news information). It would be quite easy to modify the news() parser to parse a suitably described Markdown format. The main change from the current text parser would be to expect a prefix on the line introducing each version or category. I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. I could add it to Rtools on Windows, but I think platforms that normally install tools from source will have more trouble, because it has an unusual prerequisite (Haskell). We could fall back to displaying the NEWS.md file without processing, but that makes lots of code more complicated. pandoc is only viable on platforms with a pre-compiled binary, which does not even cover all the CRAN check platforms. (It is not easy to compile ab initio from source even on a mainstream Linux, but they mostly have pandoc binaries.) The next problem is that using pandoc as the de facto standard for .md depends on the version of pandoc, and .md seems to have no way to specify the version required. (If it does, people are failing to use it ) -- Brian D. Ripley, rip...@stats.ox.ac.uk Emeritus Professor of Applied Statistics, University of Oxford 1 South Parks Road, Oxford OX1 3TG, UK __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 23 May 2015 at 08:14, Duncan Murdoch wrote: | I think the harder problem is display. CRAN can run pandoc, but can | users who install the package from source? I would expect some obscure | platforms (like Windows ;-) would not have it available. I could add it | to Rtools on Windows, but I think platforms that normally install tools | from source will have more trouble, because it has an unusual | prerequisite (Haskell). We could fall back to displaying the NEWS.md | file without processing, but that makes lots of code more complicated. If you had spare time, you could coordinate with JJ. He somehow managed to get pandoc to build as a standalone binary without any runtime requirements so that it could be included with RStudio. And so it has been for quite some time. Every machine with an RStudio installation has pandoc. Maybe we could consider carrying this over from RStudio into Rtools? Similarly, on OS X and Linux maybe we could add a layer that would R allow to query an RStudio instance, if present, to tell it where its pandoc is, if no suitable pandoc binary is in the path. Just thinking out loud... Dirk -- http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. Gabor [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
I agree. It is not worth all the trouble just to save the bit of hassle to go to the package's Github site. In fact, the release notes on Github are more meaningful than a plain text NEWS.md or even a converted NEWS.html from Pandoc, e.g. you can include bug report numbers and attribute to users by @username (they all have hyperlinks attached on them, so it is easy to see more details of bugs/features if one really cares). Personally, I feel it is very worthwhile going to Github and reading the release notes there. I'd be unhappy with converting NEWS.md to NEWS.html by Pandoc. I know not all people use Github, but I feel if a package author has a NEWS.md, chances are this package is on Github. Re Kurt's analysis of NEWS.md on CRAN, I guess that is because R CMD check will warn against NEWS.md at the top level. I know a lot of packages on Github have the NEWS.md file, and it has been removed from the tarball to make R CMD check happy. Regards, Yihui -- Yihui Xie xieyi...@gmail.com Web: http://yihui.name On Sat, May 23, 2015 at 8:25 AM, Gábor Csárdi csardi.ga...@gmail.com wrote: On Sat, May 23, 2015 at 8:14 AM, Duncan Murdoch murdoch.dun...@gmail.com wrote: [...] I think the harder problem is display. CRAN can run pandoc, but can users who install the package from source? I would expect some obscure platforms (like Windows ;-) would not have it available. [...] I don't think pandoc is the best way to go with NEWS.md (and README.md, actually). I would be surprised if many package maintainer built their NEWS/README files with pandoc. They just look at them at GitHub (or another similar service). GitHub has API for building HTML from MarkDown: https://developer.github.com/v3/markdown/ It can build GitHub-flavored MarkDown, in which case you get links to GitHub issues, etc. or just plain MarkDown, like a GitHub README. If you don't want to rely on their service, then there are a multitude of lightweight MarkDown parsers available, e.g. https://github.com/markdown-it/markdown-it is a good one IMO. Pandoc is great for vignettes, but you don't need its full power for READMEs and especially not for NEWS files. In fact most NEWS.md files look good as text. Gabor __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On Sat, May 23, 2015 at 11:31 AM, Yihui Xie x...@yihui.name wrote: I agree. It is not worth all the trouble just to save the bit of hassle to go to the package's Github site. In fact, the release notes on Github are more meaningful than a plain text NEWS.md or even a converted NEWS.html from Pandoc, e.g. you can include bug report numbers and attribute to users by @username (they all have hyperlinks attached on them, so it is easy to see more details of bugs/features if one really cares). Indeed. Although the GitHub API I linked can build a HTML that includes these links, and that's already pretty good. [...] Re Kurt's analysis of NEWS.md on CRAN, I guess that is because R CMD check will warn against NEWS.md at the top level. I know a lot of packages on Github have the NEWS.md file, and it has been removed from the tarball to make R CMD check happy. I agree. The only package with inst/NEWS.md on CRAN is in fact mine (crayon), and I put NEWS.md (and README.md) in inst/ so that 1) R CMD check and CRAN are both happy (CRAN was unhappy about non-valid HTML build by Pandoc from README.md), and 2) they are included in the package. Gabor [[alternative HTML version deleted]] __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
What about treating it as a plain text file (i.e. no need for CRAN to support parsing)? On 23 May 2015, at 11:08 am, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] NEWS.md support on CRAN
What I do is to use inst/NEWS.Rd as a placeholder that points to the NEWS.md on Github, e.g. http://cran.rstudio.com/web/packages/knitr/index.html Regards, Yihui -- Yihui Xie xieyi...@gmail.com Web: http://yihui.name On Fri, May 22, 2015 at 8:08 PM, Duncan Murdoch murdoch.dun...@gmail.com wrote: On 22/05/2015 8:49 PM, Imanuel Costigan wrote: Are there any plans for CRAN to support NEWS files in markdown? Bit of a hassle to go the the package’s Github (or other like) site to read NEWS. Not as far as I know. There have been discussions about increasing the support of Markdown, but so far the conclusion has been that it's too hard to do -- the support is not stable enough on all the platforms where R runs. Markdown is allowed for vignettes (because the package author processes those), so I'd suggest putting your news into a vignette instead of a news file. Put in a token news file that points to the vignette so users can find it. Duncan Murdoch __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
