On Friday, 15 October 2021 at 20:41:36 UTC, Selim Ozel wrote:
I am simply trying to unzip a compressed zip file slightly over
1GB. The de-compressed size is about 4 GB.
The code is very similar to what's explained in the
documentation [1] and it works for smaller files.
Anyone has a solution? Memory mapping [2] previously solved
some part of my issue but expand is still throwing memory
allocation failure.
Selim
[1] https://dlang.org/phobos/std_zip.html
[2]
https://forum.dlang.org/thread/[email protected]
It turns out my computer was literally running out of memory as
the file was getting unzipped. For some reason to uncompress a
1-gig file with uncompressed size of 4-gig, Zip Archive of D-Lang
tries to use more than 16 gig of RAM. I don't know why. Maybe I
missed something. I use a Windows 10, DMD v2.091 with x86_mscoff.
My work around was to call 7z from D Lang and do the compression
over there. That worked like a charm.
It seems that zip.d [1] calls uncompress routine from zlib.d [2].
Would calling zlib uncompress by chunks solve this memory issue?
Any ideas?
S
[1] https://github.com/dlang/phobos/blob/master/std/zip.d
[2] https://github.com/dlang/phobos/blob/master/std/zlib.d
