Re: unsafe caching of .fmt files
On Sun, Mar 8, 2020 at 11:56 PM Karl Berry wrote: > > Hi Han-Wen - neither fmt generation nor any other runtime generation has > ever been parallel-safe. Even if I added locking to mktexfmt (I guess > you are making use of the dynamic fmt creation), something I am not at > all anxious to do, the failed simultaneous runs that would result > doesn't sound like a good outcome for you. I'm trying to suggest write fmt to temp file; close rename temp file to destination in a parallel situation, this will do double work, but the final cached file will be correct. > Instead, I think you need to generate the formats you need before > starting the parallelism, either with multiple mktexfmt foo.fmt calls or > fmtutil-user (or fmtutil-sys, but if you want ~/.texlive2019 > to be used instead of /usr/local/texlive/... or whatever, that sounds > like fmtutil-user), depending on your needs. Thanks for the advice, but I can work around it. It did cost me an hour or so to debug, and I am reporting this problem so others don't have to spend the same amount of time. -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
Re: unsafe caching of .fmt files
Hi Han-Wen - neither fmt generation nor any other runtime generation has ever been parallel-safe. Even if I added locking to mktexfmt (I guess you are making use of the dynamic fmt creation), something I am not at all anxious to do, the failed simultaneous runs that would result doesn't sound like a good outcome for you. Instead, I think you need to generate the formats you need before starting the parallelism, either with multiple mktexfmt foo.fmt calls or fmtutil-user (or fmtutil-sys, but if you want ~/.texlive2019 to be used instead of /usr/local/texlive/... or whatever, that sounds like fmtutil-user), depending on your needs. Indeed, in this situation, I also suggest that you explicitly disable any mktex* creation. MKTEXFMT=0; export MKTEXFMT; MKTEXTFM=0; export MKTEXTFM; MKTEXMF=0; export MKTEXMF; MKTEXPK=0; export MKTEXPK Alternative: you could set TEXMFVAR or TEXMFSYSVAR to a per-process directory. But then every TeX run will need to create its own .fmt, which sounds like an awful waste of time. --best, karl.
unsafe caching of .fmt files
Hi there, while building the LilyPond regression test in clean container with parallelism, I am seeing file corruption errors from reading back TeX .fmt files. AFAICT, the various flavors of TeX cache .fmt files in ~/.texlive2019. This file is re-created when it doesn't exist, but no care is taken multiple parallel invocations of the same program do not tread on each others feet. Either some invocations see a half-finished version of the file, or two invocations write content to the same file. -- Han-Wen Nienhuys - hanw...@gmail.com - http://www.xs4all.nl/~hanwen
