Re: [oi-dev] PR #184: Support for epub and pandoc version problems
On Mon, 07 Jun 2021 19:19:35 +0200 benn wrote: > Hi, > Didn't get any time over the weekend until now. > > I've just cleaned up my epub branch, but PR #184 was automatically > closed, so I generated a new PR #186, which is just PR #184 with my > very old merge conflicts hopefully correctly resolved. Something seems to have gone wrong the new PR #186. It seems GitHub wants to merge 109 commits instead of just the latest one (where you've added epub) and some conflicts seem to have appeared. I think this might be because of how the branch was fast forwarded. It could probably be fixed by making a new clone of the latest oi-docs master and adding your changes to a new branch. I can see what the changes are and I already tested them last week, so I can make a PR myself which includes them if you're happy with that. I'm not sure why the old PR was closed. > > I'm not opposed to keeping support for older versions of pandoc if > > we think it's worthwhile. My main concern was the general quality > > of the output, it's unfortunate that changes to formatting will not > > be reflected in that case. The best solution to this would be, I > > think, to provide already compiled PDFs & epub as downloads from > > the docs website. This way only those who wished to make their own > > changes would need to run pandoc themselves - thus avoiding > > difficulties for a general user in getting hold of the correct > > versions of program. > Yes, a very good idea. Ultimately, users will not require the source, > only a button for html, dvi, pdf, epub, ... but we've a long way to > go yet. I don't think it would be too hard. In theory GitHub actions or similar CI/CD should be able to generate these just as they would binaries. In practice though installing dependencies such as texlive may be an issue. Certainly worth exploring. > > Changing the pandoc PDF formatting to be closer to the OpenSolaris > > style-guide is probably possible. This could be done with separate > > yaml and lua pandoc config and potentially an option in the script > > to chose the PDF formatting style. > No problem, I can add an option. > Once I get some time, I'll replace the Bash script with a Python > version and add a few things in the process like logging, tests, ... I agree that Python might be more suitable, although I think bash is probably ok for what we're doing at the moment. > > As it is now, the PDF output is > > presentable and look similar to the web docs (when the latest > > pandoc is used) and without the latest pandoc it's at least > > readable. For the time being, it would be simple to write up some > > information on how to pipe the markdown through pandoc, xsltproc > > and fop to get the example PDF above which is closer to the > > OpenSolaris doc style - without actually including scripts to do > > this. > Maybe if we were to add the options required to generate the > OpenSolaris style to makepdf.sh -s OpenIndiana, -s OpenSolaris, > whereby OpenIndiana would be default? I could do this, just send me > the pandoc command line options, and yaml configuration file. That would work fine. I made some progress over the weekend on replicating the OpenSolaris style (officially SolBook) in LaTex. It's not quite the same as the example PDF in my last email (page breaks are used, plus a title page). This is actually quite difficult to achieve in LaTex due to the offset paragraphs. It can be done though and I'll hopefully have it finished later this week. -- Regards, James ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] PR #184: Support for epub and pandoc version problems
On Mon, 07 Jun 2021 22:26:07 +0200 benn wrote: > > I can see what the changes are and I already tested them last week, > > so I can make a PR myself which includes them if you're happy with > > that. I'm not sure why the old PR was closed. > Go ahead. No more work this evening for me. I've merged this as #187. I've made a change to the version check to use bash regex groups for the Major and Minor version numbers. Pandoc versions go above a minor version of 9 and don't always have a patch version, so the check of >230 does not work for some versions (e.g. v2.13 was seen as 213 due to having no patch version). -- Regards, James ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] PR #184: Support for epub and pandoc version problems
Hi, On Mon, 2021-06-07 at 19:37 +0100, James Madgwick wrote: > On Mon, 07 Jun 2021 19:19:35 +0200 > benn wrote: > > > Hi, > > Didn't get any time over the weekend until now. > > > > I've just cleaned up my epub branch, but PR #184 was automatically > > closed, so I generated a new PR #186, which is just PR #184 with my > > very old merge conflicts hopefully correctly resolved. > > Something seems to have gone wrong the new PR #186. It seems GitHub > wants to merge 109 commits instead of just the latest one (where you've > added epub) and some conflicts seem to have appeared. I think this might > be because of how the branch was fast forwarded. It could probably be > fixed by making a new clone of the latest oi-docs master and adding your > changes to a new branch. > > I can see what the changes are and I already tested them last week, so > I can make a PR myself which includes them if you're happy with that. > I'm not sure why the old PR was closed. Go ahead. No more work this evening for me. > > > I'm not opposed to keeping support for older versions of pandoc if > > > we think it's worthwhile. My main concern was the general quality > > > of the output, it's unfortunate that changes to formatting will not > > > be reflected in that case. The best solution to this would be, I > > > think, to provide already compiled PDFs & epub as downloads from > > > the docs website. This way only those who wished to make their own > > > changes would need to run pandoc themselves - thus avoiding > > > difficulties for a general user in getting hold of the correct > > > versions of program. > > > > Yes, a very good idea. Ultimately, users will not require the source, > > only a button for html, dvi, pdf, epub, ... but we've a long way to > > go yet. > > I don't think it would be too hard. In theory GitHub actions or > similar CI/CD should be able to generate these just as they would > binaries. In practice though installing dependencies such as texlive > may be an issue. Certainly worth exploring. > > > > Changing the pandoc PDF formatting to be closer to the OpenSolaris > > > style-guide is probably possible. This could be done with separate > > > yaml and lua pandoc config and potentially an option in the script > > > to chose the PDF formatting style. > > > > No problem, I can add an option. > > Once I get some time, I'll replace the Bash script with a Python > > version and add a few things in the process like logging, tests, ... > > I agree that Python might be more suitable, although I think bash is > probably ok for what we're doing at the moment. Agreed. Experience, however, reveals that as we keep adding just one more small feature, we have a problem in the asymptote. But lets save the time until it is needed. > > > As it is now, the PDF output is > > > presentable and look similar to the web docs (when the latest > > > pandoc is used) and without the latest pandoc it's at least > > > readable. For the time being, it would be simple to write up some > > > information on how to pipe the markdown through pandoc, xsltproc > > > and fop to get the example PDF above which is closer to the > > > OpenSolaris doc style - without actually including scripts to do > > > this. > > > > Maybe if we were to add the options required to generate the > > OpenSolaris style to makepdf.sh -s OpenIndiana, -s OpenSolaris, > > whereby OpenIndiana would be default? I could do this, just send me > > the pandoc command line options, and yaml configuration file. > > That would work fine. I made some progress over the weekend on > replicating the OpenSolaris style (officially SolBook) in LaTex. It's > not quite the same as the example PDF in my last email (page breaks are > used, plus a title page). This is actually quite difficult to achieve > in LaTex due to the offset paragraphs. It can be done though and I'll > hopefully have it finished later this week. Great, looking forward to this. While the style is really something for the future, i.e., not the highest of priority, I personally still think it is important. Cheers Benny ___ oi-dev mailing list oi-dev@openindiana.org https://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] PR #184: Support for epub and pandoc version problems
Hi, Didn't get any time over the weekend until now. I've just cleaned up my epub branch, but PR #184 was automatically closed, so I generated a new PR #186, which is just PR #184 with my very old merge conflicts hopefully correctly resolved. The makepdf should select your newer pdf generation mechanism if a newer pandoc version is old, and still work for older pandoc version. At some stage, we can deprecate this older generation mechanism, i.e., once Debian stable supports it. There might be confusion in the future with various styles flying about the place. On Fri, 2021-06-04 at 21:21 +0100, James Madgwick wrote: > On Fri, 04 Jun 2021 11:49:20 +0200 > benn wrote: > > > > Only after some struggling with errors in the Lua filter did I > > > realize the features needed to make the PDF presentable required > > > the latest version of pandoc, or at least a version newer than the > > > one found in Debian stable. I installed the deb directly from the > > > pandoc GitHub releases. > > > > I think, I'm very much open to correction here, the settings in yaml > > (or xml) configuration files, can also be passed as command line > > options to the older pandoc version (i.e., 2.2.1). > > Indeed, the version in debian stable (which is 2.2.1) does take command > line options but not for all the options in the yaml. The most > important part of the yaml is the 'header-includes' field (which adds > LaTex commands for packages, fonts, etc) which I could not get to work > correctly when passed as a command line option. The yaml could instead > be included as a header in the markdown document itself, but this would > require it to be duplicated in every file in the docs and may not be > compatible with mkdocs. > > The Lua filters error problem I mentioned relates to the ':insert' > function in the Pandoc Lua API, which is relatively new. This option > is not present in version 2.2.1, and although I believe there's a > workaround for this particular issue, there may be other features > missing from the earlier API version. > > > > > The generation of the epub format of the manuals provided, in the > > > > past, output that was sufficient for me to proof read. Alas with > > > > the newer versions of pandoc I have not been able to review > > > > this. > > > > > > Using the latest version of pandoc, the epubs appeared to be ok, > > > though I have no experience with the format. > > > > Producing the appropriate format is important, difficult and time > > consuming. The OpenSolaris produced a very nice guide indeed called > > 'Documentation Style Guide for OpenSolaris.' It is over 250 pages > > long and contains a wealth of information. (It is generally available > > somewhere, > > It's linked from this page on the docs > (https://docs.openindiana.org/contrib/style/). > > > > > Now to emulate this style, if the more recent pandoc version is > > easier for you, great. I'll get makepdf to work for you. Anyway, it > > must be a goal with OI to support more recent versions of software > > and you are, currently, the only one tweaking the pdf style > > However, my old pdf generation mechanism was broken and I cannot > > easily update to a newer version yet. Agreed, the old mechanism will > > have to die, but killing Debian (or any other platform plagued) users > > ought to be avoided. Think of the unfortunate Gentoo Linux user > > having to recompile pandoc just cause something they do not really > > need breaks! So the old pdf generation mechanism will have to become > > deprecated; but not quite yet. > > I'm not opposed to keeping support for older versions of pandoc if we > think it's worthwhile. My main concern was the general quality of the > output, it's unfortunate that changes to formatting will not be > reflected in that case. The best solution to this would be, I think, to > provide already compiled PDFs & epub as downloads from the docs > website. This way only those who wished to make their own changes would > need to run pandoc themselves - thus avoiding difficulties for a general > user in getting hold of the correct versions of program. Yes, a very good idea. Ultimately, users will not require the source, only a button for html, dvi, pdf, epub, ... but we've a long way to go yet. > > > > The quality of the pdf produced needs to be improved and the > > > > quality of epub has still to be examined. > > > > > > The PDF is quite ok with the latest pandoc I would say (example: > > > https://github.com/OpenIndiana/oi-docs/files/6566753/getting-started.pdf). > > > It could do with a few tweaks to the LaTex, but nothing much is > > > needed urgently. Unlike LaTex I don't know about epub, it does seem > > > like there are extra settings which can be tweaked in padoc, but > > > far fewer than for LaTex/PDF. > > > > The epub was, like the pdf, there for me, my personal use without any > > tweaking. It has proved to be exceptionally handy. This needs serious > > tweaking to coincide with the OpenSolaris
