On 12/3/2017 10:19 AM, Knut Petersen wrote:
Am 01.12.2017 um 15:44 schrieb Ulrike Fischer:
This has also already been discussed in tex.sx
https://tex.stackexchange.com/questions/258127/why-lualatex-gives-erorr-too-many-open-files-but-pdflatex-does-not-on-same-lat
Thanks for that link.
I tested pdflatex with a 100.000 lines input and found that pdftex
correctly included all the 14.000+ pdfs it was expected to include.
So
* pdflatex works correctly,
* xelatex breaks after 5.000 included pdfs,
* lualatex breaks after whatever the soft limit for file descriptors
is set to.
luatex is the only engine that keeps all included pdfs open, and this is
known since at least 2015.
I think this is a serious regression compared to pdflatex.
When an image is included a reference node is put on the page and as the
dimensions have to be known the image is analyzed. The (pdf) file is
then kept open.
When a page is shipped out the part of the opened file that makes sense
in included. This can be any (number of) page(s).
When the image is inluded at the tex end the file is kept open. In
pdftex \immediate is (or at least was) the drive to force a close.
When the image is included at the lua end then file is closed unless
flagges as to be kept open.
As luatex can be used to include multiple pages it makes sense to keep
the file open. Inlcuding 1000 pages from a file takes 35 seconds when we
close the file, and 1 second when we keep it open. So, we keep it open.
We've been experimenting a bit and now consider the following. We always
close the file after a page has been included. Of course when one puts
100K images on one page one still can hit the limits of the operating
system but such it is. (I'm not sure what pdftex does when one puts 100K
images on one page.)
In addition there will be an option to keep the file open at the tex
end. So in the end we will have
\immediate\useimageresource{foo.pdf}%
\saveimageresource \lastsavedimageresourceindex % closed
\useimageresource{foo.pdf}%
\saveimageresource \lastsavedimageresourceindex %
kept open
\useimageresource{foo.pdf}%
\saveimageresource keepopen\lastsavedimageresourceindex %
kept open
\directlua{img.write(img.scan{ file = "foo.pdf" })} %
closed
\directlua{img.write(img.scan{ file = "foo.pdf", keepopen = true })} %
kept open
Of course when you want speed in including more than a handful of pages
from the same file you're off worse. But that's probbably better than
this topic popping up every few years. Of course using a mixed approach
for the same file is not our concern.
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
