On 11/28/22 8:05 AM, Marek Marczykowski-Górecki wrote: > On Tue, Nov 01, 2022 at 12:45:33PM +0100, mm wrote: >> Hi Marek, hi Andrew, hi Tobias, > >> Marek, I merged your pull request, also merged your changes into master and >> added some enhancements and created a pull request. > >> Here you could review it >> https://github.com/maiska/qubes-translation-utilz/pull/2 > >> Either way, one could try it out, there should be no more undefined or >> unknown label warnings when rendering the rst docs >> and thus could safely imho go and convert the documentation and host it on >> RTD. > >> Here is a README.md instead of spamming via email >> https://github.com/maiska/qubes-translation-utilz/blob/master/README.md > >> the configuration should be more readable also: > >> https://github.com/maiska/qubes-translation-utilz/blob/master/md2rst/config.toml > >> Please let me know if I have to fix further or this suffices. > > I think the current version of scripts is good :) Thanks! > > There are sphinx warning remaining, but there are not that many of them > and can be fixed manually (things like underscores for headers, or > single space at the beginning of line interpreted as unexpected > indentation). > > The only other issue I consider a blocker (but hopefully should be easy > to fix) is index.rst. The sidebar index is great, but in its current > form it's hard to use - it's very long. I added section captions there > and it helped a bit, but I think those should be folded by default > (similar folding like sections of the current page). I tried setting > `collapse_navigation`[1], but it didn't work. Any other ideas? > > BTW, while looking for related option, I found you can use `:glob:` > in toctree, to add everything from given subdirectory. This way we can have > some section filled automatically (as long as directory structure > reflects sections, but it is the case in most cases). > > I uploaded current version to > https://qubes-doc-rst.readthedocs.io/en/latest/ (I haven't fixed all > sphinx warnings there, so few places may have wrong indentation). I > haven't updated translated pages - those are still from previous > iteration. > Source of the above is at https://github.com/marmarek/qubes-doc/tree/work4/ > > Another thing that IMO would be better, is to keep the "introduction" page > as part of the main website, and only link to it from the documentation. > Should be easy with the redirects extension (as already done for HCL and > few others). > > The next question is what to do about qubes-doc repo. We can either > replace the current content with converted to rst (and keep markdown in > a separate branch, for historical reasons), or create new one > (qubes-doc-rst?), and basically archive qubes-doc. >
Git already keeps everything for historical reasons, so I don't think it's necessary to keep a separate branch around *forever* just for historical reasons, which would create extra clutter and get in the way, but keeping the historical branch for, say, a year would make sense. For a permanent marker, it seems cleaner just use a tag to indicate where the transition occurred. As far as I can tell, the main difference between reusing the same repo and creating a new repo is simply that, in the latter case, history would be split across two repos, and we'd have to update all of our references to "qubes-doc" everywhere. So, I'm inclined to say we should keep using the existing repo. > In any case, the _doc submodule should be replaced with generated > redirects to the new documentation. IMO it can be put into > qubesos.github.io repository directly, as it shouldn't really change > (it's only to keep existing documentation links on mailing list, forum > etc working, shouldn't be used for new content). > > [1] https://sphinx-rtd-theme.readthedocs.io/en/stable/configuring.html > >> Otherwise I will briefly handson evaluate weblate locally and report back. > > Do you have any update on weblate? > >> @Andrew, > >> regarding the minimal debian jekyll vm, here is a way with a template and an >> app vm. > >> qvm-clone debian-11-minimal jekyll-tvm > >> in jekyll-tvm: >> apt install qubes-core-agent-networking >> apt install ruby-full build-essential zlib1g-dev vim >> apt install qubes-core-agent-passwordless-root >> apt install firefox-esr git > >> in jekyll-app-vm: >> echo '# Install Ruby Gems to ~/gems' >> ~/.bashrc >> echo 'export GEM_HOME="$HOME/gems"' >> ~/.bashrc >> echo 'export PATH="$HOME/gems/bin:$PATH"' >> ~/.bashrc >> source ~/.bashrc >> gem install jekyll bundler >> find . -name gem # '/home/user/.local/share/gem/' >> bundle config set --local path '/home/user/.local/share/gem/' >> git clone -b new-master --recursive >> https://github.com/QubesOS/qubesos.github.io.git; cd qubesos.github.io.rtd/ >> bundle install >> bundle exec jekyll serve --incremental > > > >> All the best, > >> m > > > >> On 10/4/22 12:29, Marek Marczykowski-Górecki wrote: >>> Hi, >>> >>> Thanks for the update. Don't worry about the schedule - while we do want >>> to finish this soon, it doesn't need to be yesterday. >>> >>> >>> On Tue, Oct 04, 2022 at 06:54:45AM +0200, mm wrote: >>>> Hello Marek, hello Andrew, >>> >>>> I can have spare time starting Sunday to focus & reply. >>> >>>> (Please, Marek, recall that i have pointed out mid September that I am >>> out off the map >>> >>>> basically from 15.09 till 8.10.) >>> >>>> Nevertheless tried to fix different issues in some spare time, such as >>> embedding videos, toctree etc. >>> >>>> I will push my final version on master asap. (Still haven't gotten to >>> the PR/merge/refactor), suggest to wait till Sunday. >>> >>>> If that is not possible there are several issues to be considered: >>> >>>> 0. please pay attention to config.toml and conf.py from master and the >>> files in preparation. >>> >>>> 1. which rst files are to be copied (there are 3 atm, since the most >>> issues should be fixed via rstwriter2), which to be removed, >>> >>>> different extensions to be configured such as redirects (I suggest >>> leaving an empty hcl.rst (downloads etc) with a redirect, to be >>> populated in a >>> >>>> later stage via jinja scripting), new strikeout extension and new >>> role, video directives, which markdown files are also to be copied >>>> - is doc.md the right one etc? latex documents are rendered correctly >>> (toctree req) >>> >>> I took toctree-example.rst and manually synced with recent doc.md. It >>> seems to work, including PDF version. The PDF could use explicit TOC at >>> the beginning, but it is in PDF metadata, so can be seen in the side >>> panel of PDF viewer. >>> >>> >>>> 2. !! Problem that should have a solution: >>> >>>> f.ex.https://qubes-doc-rst.readthedocs.io/en/latest/user/troubleshooting/installation-troubleshooting.html >>> >>>> in the sentence: >>> >>>> "Here are the steps to fix this. Note that this allows sys-net and >>> sys-usb to take complete control of the system, as described in the FAQ >>> here:" >>>> there should be a cross reference to FAQ VT-d iommu section but is >>> not. (with my version this is also a problem) >>> >>>> Unfortunately the documentation (sphinx referencing via roles, perhaps >>> the implementation?) is not saying anything how to handle punctuation >>> with sections and cross ref with ref, >>> >>>> so as of the moment I do not have an idea, apart from the basic stuff. >>> >>> Yes, I hit similar issues elsewhere. Most of issues like this I fixed by >>> importing objects.inv and taking references from there (this is also >>> pushed to the PR), but there are still few corner cases like the noted >>> above. It seems '&' is handled differently by markdown and sphinx. All >>> the sphinx warnings fit for me on one screen, so I think those few >>> remaining cases can be fixed manually, there is no need to make the >>> converter perfect. >>> >>> >>>> I also tried explicit automatic labeling above each section with a >>> directive (.. _id-of-section) - did not work out >>> >>>> 3. different new files to be considered: >>>> f.e.x 2 versions of the privacy content, how to edit the documentation >>> - markdown & rst version, visual style guide etc >>> >>> Yes, I left some of them on the old site (with matching redirects), but >>> this needs to be sorted out. >>> >>>> 4. what is done so far on master >>> >>>> -- video directives >>> >>>> -- toctree index, notes, schedules, upgrade, how to restore >>> >>>> -- section cross referencing small fixes >>> >>>> -- svg conversion to png for selected files and replacement in rst docs >>> >>>> -- convert leftover markdown links in rst files >>> >>>> -- markdown redirects >>> >>>> -- added directives for warning & note (can be used also for >>> indicating translated content) >>>> -- anything i forgot >>> >>>> 5. weblate - localization platform should be the way to go imho (sry >>> not a big fan of transifex rn, weblate is OS etc, >>>> some more objective analysis will follow), there are several issues to >>> be clarified, wip, tails guys need good questions to work with >>>> - will be done on sunday >>>> 6. anything i forgot, did not address, mentioned too briefly or not >>> clarified i will address on sun >>>> (if it is still needed) >>> >>>> all the best, >>>> m >>> >>>> On 10/2/22 18:24, Marek Marczykowski-Górecki wrote: >>>>> On Tue, Sep 27, 2022 at 01:15:56AM +0200, Marek Marczykowski-Górecki >>>>> wrote: >>>>>> On Mon, Sep 26, 2022 at 11:33:22PM +0200, mm wrote: >>>>>>> Hi Marek, >>>>>>> >>>>>>> >>>>>>> On 9/26/22 00:01, Marek Marczykowski-Górecki wrote: >>>>>>>> Hi M, >>>>>>>> >>>>>>>> In fact, I'm working on translation-utilz right now too. Marta used >>>>> her >>>>>>>> google-foo and found this gem: >>>>>>>> https://github.com/SamWilsn/docutils-rst-writer >>>>>>>> >>>>>>> this looks very nice!! :) Thanks Marta! :) >>>>>>> >>>>>>>> So, I replaced most of the qubesrstwriter2.py with just this thing >>> and >>>>>>>> it's almost flawless (especially tables, images etc work out of the >>>>>>>> box). >>>>>>> >>>>>>> OK, thanks for the PR, I'll merge into and get rid of the obsolete >>>>> stuff >>>>>>> from my branch this days. >>>>>>> >>>>>>> I have some weird customized fixes all over, but this should work. >>>>> >>>>> I spent some more time on polishing the converter (all pushed to the >>>>> same PR), and I think it's good enough already. There are few remaining >>>>> issues that Sphinx complains about, but at least some of them look like >>>>> issues already present in the original markdown files, and can be fixed >>>>> in the RST version manually. I'd prefer it this way (for this category >>>>> of issues), because Sphinx gives much better error reporting than >>>>> jekyll, so it's easier to iterate. >>>>> >>>>> The rendered content can be seen at >>>>> https://qubes-doc-rst.readthedocs.io/en/latest/ >>>>> And the converted source at >>>>> https://github.com/marmarek/qubes-doc/tree/work3/ >>>>> >>>>> Note the above has already enabled PDF+ePUB version, and German and >>>>> Spanish translations (although most content is not translated yet). >>>>> >>>>> I did also used your markdownredirector to generate redirects to RTD, >>>>> here: >>>>> https://github.com/QubesOS/qubes-doc/pull/1272 >>>>> >>>>> It's deployed to wwwpreview.qubes-os.org. It works for most links, for >>>>> example >>>>> https://wwwpreview.qubes-os.org/doc/secondary-storage/ >>>>> >>>>> But, htmlproofer complains about links to specific sections - which >>>>> indeed are broken now ('#' part does not survive redirect). For >>> example: >>>>> https://wwwpreview.qubes-os.org/doc/#troubleshooting >>>>> >>>>> I don't think that's a big issue in practice, but will require >>> adjusting >>>>> htmlproofer call. >>>>> >>>>> Anyway, I see further steps as: >>>>> >>>>> 1. Review and merge (or reject) current PRs to qubes-doc. They will >>>>> become completely obsolete after conversion. Some of them are waiting >>>>> for my feedback... >>>>> >>>>> 2. Convert the whole qubes-doc repo to ReST format, and connect to >>>>> readthedocs.org as doc.qubes-os.org (any better idea for the domain?) >>>>> >>>>> 3. Disconnect qubes-doc submodule from qubesos.github.io repo, and >>>>> replace with generated redirects (the content of PR 1272 above). >>>>> >>>>> 4. Incrementally fix remaining issues in rst files (at this point, I >>>>> don't see any critical ones). Some I see Maya fixed already in the >>>>> fixed-rst-errors dir. >>>>> >>>>> 5. Adjust documentation contributing guidelines for the new format >>> (this >>>>> page actually has rst format issues after conversion, but I haven't >>>>> bothered to fix them, because the content would need significant change >>>>> anyway). >>>>> >>>>> And then, translation-related: >>>>> >>>>> 6. Cleanup our transifex project (remove previous attempts, like >>>>> doc-related markdown files) >>>>> >>>>> 7. Upload rst version (already done, but will need re-uploading after >>>>> final conversion). >>>>> >>>>> 8. Setup CI job to pull translations to qubes-translated repo (it's a >>>>> single transifex client call, I already have it ready to go from >>>>> previous attempt). >>>>> >>>>> 9. Adjust theme on translated pages to include warning that non-English >>>>> version may be less accurate. >>>>> >>>>> 10. Write/Adjust guidelines for translators, open transifex project >>>>> again. >>>>> >>>>> The above might get adjusted, if we'd like to use weblate instead, >>> but I >>>>> think that's separate discussion (both in practice require migration to >>>>> rst first). >>>>> >>>>> There is also a topic of translating the website itself, which IMO will >>>>> become much easier once documentation is moved away. >>>>> >>>>> >>> > > > -- You received this message because you are subscribed to the Google Groups "qubes-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to qubes-devel+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-devel/803adc23-492a-da44-1118-d1a4b26245a0%40qubes-os.org.
