On 5/17/2021 11:43 PM, Thomas A. Schmitz wrote:
Hi all,
I have a rather large (almost 120,000 lines) xml file, processing pulls
in another large xml file. The project compiles under mkiv; with lmtx, I
get
tex error > tex error on line 8 in file : TeX capacity exceeded,
sorry [reference count=32767]
This is not the normal memory consumption that bothers you
\enabletrackers[system.usage]
will show you more detail. Normally the default context configuration
will provide enough. In lmts memory is abit more dynamic so we start otu
smaller and then stepwise grow. And one can configure even more
(actually i just updated some configuration options for that).
Anyway: this more smells like some nesting going wrong and therefore
excessive token sharing
<macro> \reallanguagetag
just twenty pages shy of the end of the document.
There is an error log with this content:
return {
["errortype"]=3,
["filename"]="",
["lastcontext"]="\n\n\n<macro> \\reallanguagetag \n \n<macro>
\\reallanguagetag #1->\\ifcsname \\??languagelinked #1\n \\endcsname
\\lastnamedcs \\else #1\\fi \n<macro> \\setusedlanguage #1->\\edef
\\currentusedlanguage {\\reallanguagetag {#1}\n }\\ifempty
\\currentusedlanguage \\let \\currentusedlanguage \\currentlanguage
\\always\\let \\usedlanguageparameter \\languageparameter \\orelse \\ifx
\\currentusedlanguage \\v!global \\let \\currentusedlanguage\n\n<macro>
\\typo_delimited_push ...\\currentdelimitedtext {#1}\\edef
\\m_delimited_argument {#2}\\ifempty \\m_delimited_argument
\\typo_delimited_set_language_nop \n \\else
\\typo_delimited_set_language_yes \\fi \\let
\\currentparentdelimitedtext \\currentdelimitedtext \\global \\advance
\\c_typo...\n\n<macro> \\delimitedtext [#1]#*[#2]->\\dontleavehmode
\\typo_delimited_push {#1}{#2}\n \\edef \\p_delimited_method
{\\delimitedtextparameter \\c!method }\\ifx \\p_delimited_method
\\v!font \\expandafter \\typo_delimited_fontdriven \\else \\expandafter
\\typo_delimited_other \\fi \n\n<to be read again> \n
{\n...\n\n<initial> \n ",
["lastluaerror"]="",
["lasttexerror"]="TeX capacity exceeded, sorry [reference count=32767]",
["lasttexhelp"]="If you really absolutely need more capacity, you can
ask a wizard to enlarge me.",
["linenumber"]=8,
["offset"]=10,
["skiplinenumber"]=8,
}
In mkiv, these are the stats at the end of the run:
mwe ...
mkiv lua stats > used config file:
selfautoparent:/texmf/web2c/texmfcnf.lua
mkiv lua stats > used cache path:
/mnt/shared/lmtx/tex/texmf-cache/luatex-cache/context/5fe67e0bfe781ce0dde776fb1556f32e
mkiv lua stats > resource resolver: loadtime 0.009 seconds, 1 scans
with scantime 0.005 seconds, 0 shared scans, 27 found files, scanned
paths: /home/tas/texmf
mkiv lua stats > stored bytecode data: 455 modules (0.122 sec), 96
tables (0.007 sec), 551 chunks (0.130 sec)
mkiv lua stats > traced context: maxstack: 1422, freed: 2, unreachable:
1420
mkiv lua stats > cleaned up reserved nodes: 63 nodes, 9 lists of 442
mkiv lua stats > node memory usage: 17 attribute, 4 attribute_list, 1
glue, 51 glue_spec, 3 kern, 2 penalty, 2 temp, 2 user_defined
mkiv lua stats > node list callback tasks: 8 unique task lists, 9336
instances (re)created, 177413 calls
mkiv lua stats > h-node processing time: 4.800 seconds including kernel
mkiv lua stats > attribute processing time: 0.617 seconds front- and
backend
mkiv lua stats > driver time: 0.886 seconds
mkiv lua stats > used backend: pdf (backend for directly generating pdf
output)
mkiv lua stats > jobdata time: 0.161 seconds saving, 0.052 seconds loading
mkiv lua stats > callbacks: internal: 261417, file: 319, direct: 3,
late: 31831, function 200128, total: 493698 (464 per page)
mkiv lua stats > randomizer: resumed with value 0.47143577262518
mkiv lua stats > positions: 9535 collected, 9535 regular, 0 special
mkiv lua stats > v-node processing time: 0.401 seconds
mkiv lua stats > result saved in file: plutarchusvitae.pdf,
compresslevel 3, objectcompresslevel 1
mkiv lua stats > loaded patterns: de::3 en::1 la::65, load time: 0.000
mkiv lua stats > xml load time: 0.648 seconds, 4 files, 4 converted
mkiv lua stats > lxml preparation time: 0.626 seconds, 233814 nodes,
578788 lpath calls, 578743 cached calls
mkiv lua stats > interactive elements: 0 references, 37 destinations
mkiv lua stats > margin data: 10798 entries, 0 pending
mkiv lua stats > loaded fonts: 9 files: garamondpremrpro.otf,
sabonltstd-bold.otf, sabonltstd-italic.otf, sabonltstd-roman.otf,
kadmosu.otf, oldstandard-bold.otf, oldstandard-italic.otf,
xitsmath-regular.otf, dejavusansmono.ttf
mkiv lua stats > font engine: otf 3.113, afm 1.513, tfm 1.000, 17
instances, 6 shared in backend, 0 common vectors, 6 common hashes, load
time 0.146 seconds
mkiv lua stats > used platform: linux-64, type: unix, binary subtree:
texmf-linux-64
mkiv lua stats > used engine: luatex version: 1.13, functionality
level: 7420, banner: this is luatex, version 1.13.0 (tex live 2021)
mkiv lua stats > used hash slots: 47131 of 65536 + 250000
mkiv lua stats > lua properties: engine: lua 5.3, used memory: 601 MB,
ctx: 579 MB, max: unknown MB, hash chars: min(64,40), symbol mask: utf
(τεχ)
mkiv lua stats > runtime: 22.310 seconds, 1068 processed pages, 1068
shipped pages, 47.871 pages/second
system | total runtime: 213.928 seconds
hardly demanding in terms of memory
Hans
-----------------------------------------------------------------
Hans Hagen | PRAGMA ADE
Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-----------------------------------------------------------------
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage : http://www.pragma-ade.nl / http://context.aanhet.net
archive : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___________________________________________________________________________________
