If you're not curious about the background, skip to the part below starting with "The main problem". In short, I want to learn more about some code to try to make it thread-safe.
The last few days I've been experimenting with using parallelization in part of
LyX's code.
I'm currently using a for_each loop inside of latexParagraphs() to parallelize
over TeXOnePar(). I'm shocked that it actually runs in parallel. The reason I'm
shocked is because I don't know what I'm doing. My current code uses a lot of
ugly hacks and is not usable for anything other than curiosity. If you really
want to see it, I can share it. But I really wouldn't recommend it :)
I used the hacks just to get an idea of whether such parallelization would
actually lead to a speed-up. It does. For example, on an 8-core machine, the
document files (user guide, etc.) export to .tex about 4x faster. This isn't
that helpful from a user perspective, because the main bottle-neck is compiling
to PDF, not the LaTeX generation. But it's still cool to see that it actually
does have a real effect. I think the speedup will be even faster if we could
make Paragraph::latex() thread-safe. Currently I protect part of the code with
a mutex, which I ask a question about below.
I don't think my work is likely to be ready for upstream for a while, and even
then I'm not sure what others would think. In the short-run I'm motivated by a
personal use case, which is that I have a document with 20 children and each of
those children takes considerable time to be exported to TeX because they each
use knitr. I would like to do those exports in parallel. I don't imagine this
is a common use case.
The main problem I currently face is the following code inside
Paragraph::latex():
d->latexInset(bparams, os, rp, running_font,
basefont, real_outerfont, open_font,
runningChange, style, i, column, fontswitch_inset,
closeLanguage, (lang_switched_at_inset || langClosed));
If I protect it with a mutex, then things work much better. But that's exactly
the code that I need to run in parallel for my knitr child documents to export
in parallel. Could someone explain intuitively why this code is not
thread-safe? Is there any hope of making it thread-safe without major surgery?
In addition to that code, there's other parts that are not thread-safe. I know
this because when I export to LaTeX, LyX crashes once every 5 times. I haven't
put much effort into finding the other parts, but I think I could make progress
if we could address the d->latexInset() issue I ask about above.
Thanks,
Scott
signature.asc
Description: PGP signature
-- lyx-devel mailing list lyx-devel@lists.lyx.org http://lists.lyx.org/mailman/listinfo/lyx-devel
