-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 On Tue, Nov 01, 2022 at 12:45:33PM +0100, mm wrote: > Hi Marek, hi Andrew, hi Tobias, > > Marek, I merged your pull request, also merged your changes into master and > added some enhancements and created a pull request. > > Here you could review it > https://github.com/maiska/qubes-translation-utilz/pull/2 > > Either way, one could try it out, there should be no more undefined or > unknown label warnings when rendering the rst docs > and thus could safely imho go and convert the documentation and host it on > RTD. > > Here is a README.md instead of spamming via email > https://github.com/maiska/qubes-translation-utilz/blob/master/README.md > > the configuration should be more readable also: > > https://github.com/maiska/qubes-translation-utilz/blob/master/md2rst/config.toml > > Please let me know if I have to fix further or this suffices.
I think the current version of scripts is good :) Thanks! There are sphinx warning remaining, but there are not that many of them and can be fixed manually (things like underscores for headers, or single space at the beginning of line interpreted as unexpected indentation). The only other issue I consider a blocker (but hopefully should be easy to fix) is index.rst. The sidebar index is great, but in its current form it's hard to use - it's very long. I added section captions there and it helped a bit, but I think those should be folded by default (similar folding like sections of the current page). I tried setting `collapse_navigation`[1], but it didn't work. Any other ideas? BTW, while looking for related option, I found you can use `:glob:` in toctree, to add everything from given subdirectory. This way we can have some section filled automatically (as long as directory structure reflects sections, but it is the case in most cases). I uploaded current version to https://qubes-doc-rst.readthedocs.io/en/latest/ (I haven't fixed all sphinx warnings there, so few places may have wrong indentation). I haven't updated translated pages - those are still from previous iteration. Source of the above is at https://github.com/marmarek/qubes-doc/tree/work4/ Another thing that IMO would be better, is to keep the "introduction" page as part of the main website, and only link to it from the documentation. Should be easy with the redirects extension (as already done for HCL and few others). The next question is what to do about qubes-doc repo. We can either replace the current content with converted to rst (and keep markdown in a separate branch, for historical reasons), or create new one (qubes-doc-rst?), and basically archive qubes-doc. In any case, the _doc submodule should be replaced with generated redirects to the new documentation. IMO it can be put into qubesos.github.io repository directly, as it shouldn't really change (it's only to keep existing documentation links on mailing list, forum etc working, shouldn't be used for new content). [1] https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html > Otherwise I will briefly handson evaluate weblate locally and report back. Do you have any update on weblate? > @Andrew, > > regarding the minimal debian jekyll vm, here is a way with a template and an > app vm. > > qvm-clone debian-11-minimal jekyll-tvm > > in jekyll-tvm: > apt install qubes-core-agent-networking > apt install ruby-full build-essential zlib1g-dev vim > apt install qubes-core-agent-passwordless-root > apt install firefox-esr git > > in jekyll-app-vm: > echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc > echo 'export GEM_HOME="$HOME/gems"' >> ~/.bashrc > echo 'export PATH="$HOME/gems/bin:$PATH"' >> ~/.bashrc > source ~/.bashrc > gem install jekyll bundler > find . -name gem # '/home/user/.local/share/gem/' > bundle config set --local path '/home/user/.local/share/gem/' > git clone -b new-master --recursive > https://github.com/QubesOS/qubesos.github.io.git; cd qubesos.github.io.rtd/ > bundle install > bundle exec jekyll serve --incremental > > > > All the best, > > m > > > > On 10/4/22 12:29, Marek Marczykowski-Górecki wrote: > > Hi, > > > > Thanks for the update. Don't worry about the schedule - while we do want > > to finish this soon, it doesn't need to be yesterday. > > > > > > On Tue, Oct 04, 2022 at 06:54:45AM +0200, mm wrote: > > > Hello Marek, hello Andrew, > > > > > I can have spare time starting Sunday to focus & reply. > > > > > (Please, Marek, recall that i have pointed out mid September that I am > > out off the map > > > > > basically from 15.09 till 8.10.) > > > > > Nevertheless tried to fix different issues in some spare time, such as > > embedding videos, toctree etc. > > > > > I will push my final version on master asap. (Still haven't gotten to > > the PR/merge/refactor), suggest to wait till Sunday. > > > > > If that is not possible there are several issues to be considered: > > > > > 0. please pay attention to config.toml and conf.py from master and the > > files in preparation. > > > > > 1. which rst files are to be copied (there are 3 atm, since the most > > issues should be fixed via rstwriter2), which to be removed, > > > > > different extensions to be configured such as redirects (I suggest > > leaving an empty hcl.rst (downloads etc) with a redirect, to be > > populated in a > > > > > later stage via jinja scripting), new strikeout extension and new > > role, video directives, which markdown files are also to be copied > > > - is doc.md the right one etc? latex documents are rendered correctly > > (toctree req) > > > > I took toctree-example.rst and manually synced with recent doc.md. It > > seems to work, including PDF version. The PDF could use explicit TOC at > > the beginning, but it is in PDF metadata, so can be seen in the side > > panel of PDF viewer. > > > > > > > 2. !! Problem that should have a solution: > > > > > f.ex.https://qubes-doc-rst.readthedocs.io/en/latest/user/troubleshooting/installation-troubleshooting.html > > > > > in the sentence: > > > > > "Here are the steps to fix this. Note that this allows sys-net and > > sys-usb to take complete control of the system, as described in the FAQ > > here:" > > > there should be a cross reference to FAQ VT-d iommu section but is > > not. (with my version this is also a problem) > > > > > Unfortunately the documentation (sphinx referencing via roles, perhaps > > the implementation?) is not saying anything how to handle punctuation > > with sections and cross ref with ref, > > > > > so as of the moment I do not have an idea, apart from the basic stuff. > > > > Yes, I hit similar issues elsewhere. Most of issues like this I fixed by > > importing objects.inv and taking references from there (this is also > > pushed to the PR), but there are still few corner cases like the noted > > above. It seems '&' is handled differently by markdown and sphinx. All > > the sphinx warnings fit for me on one screen, so I think those few > > remaining cases can be fixed manually, there is no need to make the > > converter perfect. > > > > > > > I also tried explicit automatic labeling above each section with a > > directive (.. _id-of-section) - did not work out > > > > > 3. different new files to be considered: > > > f.e.x 2 versions of the privacy content, how to edit the documentation > > - markdown & rst version, visual style guide etc > > > > Yes, I left some of them on the old site (with matching redirects), but > > this needs to be sorted out. > > > > > 4. what is done so far on master > > > > > -- video directives > > > > > -- toctree index, notes, schedules, upgrade, how to restore > > > > > -- section cross referencing small fixes > > > > > -- svg conversion to png for selected files and replacement in rst docs > > > > > -- convert leftover markdown links in rst files > > > > > -- markdown redirects > > > > > -- added directives for warning & note (can be used also for > > indicating translated content) > > > -- anything i forgot > > > > > 5. weblate - localization platform should be the way to go imho (sry > > not a big fan of transifex rn, weblate is OS etc, > > > some more objective analysis will follow), there are several issues to > > be clarified, wip, tails guys need good questions to work with > > > - will be done on sunday > > > 6. anything i forgot, did not address, mentioned too briefly or not > > clarified i will address on sun > > > (if it is still needed) > > > > > all the best, > > > m > > > > > On 10/2/22 18:24, Marek Marczykowski-Górecki wrote: > > >> On Tue, Sep 27, 2022 at 01:15:56AM +0200, Marek Marczykowski-Górecki > > >> wrote: > > >>> On Mon, Sep 26, 2022 at 11:33:22PM +0200, mm wrote: > > >>>> Hi Marek, > > >>>> > > >>>> > > >>>> On 9/26/22 00:01, Marek Marczykowski-Górecki wrote: > > >>>>> Hi M, > > >>>>> > > >>>>> In fact, I'm working on translation-utilz right now too. Marta used > > >> her > > >>>>> google-foo and found this gem: > > >>>>> https://github.com/SamWilsn/docutils-rst-writer > > >>>>> > > >>>> this looks very nice!! :) Thanks Marta! :) > > >>>> > > >>>>> So, I replaced most of the qubesrstwriter2.py with just this thing > > and > > >>>>> it's almost flawless (especially tables, images etc work out of the > > >>>>> box). > > >>>> > > >>>> OK, thanks for the PR, I'll merge into and get rid of the obsolete > > >> stuff > > >>>> from my branch this days. > > >>>> > > >>>> I have some weird customized fixes all over, but this should work. > > >> > > >> I spent some more time on polishing the converter (all pushed to the > > >> same PR), and I think it's good enough already. There are few remaining > > >> issues that Sphinx complains about, but at least some of them look like > > >> issues already present in the original markdown files, and can be fixed > > >> in the RST version manually. I'd prefer it this way (for this category > > >> of issues), because Sphinx gives much better error reporting than > > >> jekyll, so it's easier to iterate. > > >> > > >> The rendered content can be seen at > > >> https://qubes-doc-rst.readthedocs.io/en/latest/ > > >> And the converted source at > > >> https://github.com/marmarek/qubes-doc/tree/work3/ > > >> > > >> Note the above has already enabled PDF+ePUB version, and German and > > >> Spanish translations (although most content is not translated yet). > > >> > > >> I did also used your markdownredirector to generate redirects to RTD, > > >> here: > > >> https://github.com/QubesOS/qubes-doc/pull/1272 > > >> > > >> It's deployed to wwwpreview.qubes-os.org. It works for most links, for > > >> example > > >> https://wwwpreview.qubes-os.org/doc/secondary-storage/ > > >> > > >> But, htmlproofer complains about links to specific sections - which > > >> indeed are broken now ('#' part does not survive redirect). For > > example: > > >> https://wwwpreview.qubes-os.org/doc/#troubleshooting > > >> > > >> I don't think that's a big issue in practice, but will require > > adjusting > > >> htmlproofer call. > > >> > > >> Anyway, I see further steps as: > > >> > > >> 1. Review and merge (or reject) current PRs to qubes-doc. They will > > >> become completely obsolete after conversion. Some of them are waiting > > >> for my feedback... > > >> > > >> 2. Convert the whole qubes-doc repo to ReST format, and connect to > > >> readthedocs.org as doc.qubes-os.org (any better idea for the domain?) > > >> > > >> 3. Disconnect qubes-doc submodule from qubesos.github.io repo, and > > >> replace with generated redirects (the content of PR 1272 above). > > >> > > >> 4. Incrementally fix remaining issues in rst files (at this point, I > > >> don't see any critical ones). Some I see Maya fixed already in the > > >> fixed-rst-errors dir. > > >> > > >> 5. Adjust documentation contributing guidelines for the new format > > (this > > >> page actually has rst format issues after conversion, but I haven't > > >> bothered to fix them, because the content would need significant change > > >> anyway). > > >> > > >> And then, translation-related: > > >> > > >> 6. Cleanup our transifex project (remove previous attempts, like > > >> doc-related markdown files) > > >> > > >> 7. Upload rst version (already done, but will need re-uploading after > > >> final conversion). > > >> > > >> 8. Setup CI job to pull translations to qubes-translated repo (it's a > > >> single transifex client call, I already have it ready to go from > > >> previous attempt). > > >> > > >> 9. Adjust theme on translated pages to include warning that non-English > > >> version may be less accurate. > > >> > > >> 10. Write/Adjust guidelines for translators, open transifex project > > >> again. > > >> > > >> The above might get adjusted, if we'd like to use weblate instead, > > but I > > >> think that's separate discussion (both in practice require migration to > > >> rst first). > > >> > > >> There is also a topic of translating the website itself, which IMO will > > >> become much easier once documentation is moved away. > > >> > > >> > > > - -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmOE3F8ACgkQ24/THMrX 1yxPFwf7BeLksa45KgWFAgU2tDpIoe9Rx3YukKrtD5UtmgzUgCdQTlEUamF3OpND lqjG5zet59CvBWFCu3wx5p1XBy4Ohp2MxwX1gK+A+XOrvmOJNSIFhLabE7mEZSul DpLknfgZ/0dh1ITg3Qgd+79FkYqPaC2ulOUnfozoC6pP8aqzelu85W9WPtDPxU73 H3+MDoJ19zFExOBFa/21A9qrNNZvd97t/O9+hwTwXErXr2GdM3k8u0AfutUm2zHU saEyfnSDkR1hF2ANwCVfSnNCa/99nsA2sAnuVkwbsinTDSEBpkWcDw5PhOwBQvuz 3zFhHBZ0VuSKlDX3Wjb6sLJmhk8LZw== =sxMo -----END PGP SIGNATURE----- -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/Y4TcX87F0SEq6asp%40mail-itl.
