Mark Geisert writes: > The page fault numbers are comparable to what you've shown for Cygwin > on your system. The long pause after zstd prints "Constructing > partial suffix array" is because zstd is cpu-bound in qsort() for a > long time. No paging during that time.
Then there's probably another performance bug in qsort since that step finishes much faster on Linux too. > Then when the statistics start being printed out, that's when the > paging insanity starts. OK, so you see the same thing. Good. > What I discovered is that zstd is repeatedly asking malloc() for large > memory blocks, presumably to mmap files in, then free()ing them. Oh, so it's not recycling those… maybe that's not very hard to change. > Any malloc request 256K or larger is fulfilled by mmap() rather than > enlarging the heap for it. But crucially, there is no mechanism for > our malloc to hang on to freed mmap()ed pages for future use. If you > free an mmap()ed block, it is unmap()ed immediately. So for zstd's > usage pattern you get an incredible number of page faults to satisfy > the mmap()s and Windows seems to take a non-trivial bit of time for > each mmap(). It probably puts backing store behind it even when there's enough memory. Come to think of it I should try to move the page file to the NVMe drive (off the SSD it's on currently) and see if that changes things. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Factory and User Sound Singles for Waldorf Q+, Q and microQ: http://Synth.Stromeko.net/Downloads.html#WaldorfSounds
