On 2022/05/26 20:51, Rubén Llorente wrote: > On Sat, May 21, 2022 at 10:48:33AM +0100, Stuart Henderson wrote: > > how is it detecting how much memory to use? if it's looking at available > > physical memory without considering resource limits, could that be changed? > > > > there is precedent for programs to adjust RLIMIT_DATA from the default to > > the max available within the current limit if they know they're likely > > to need it - see e.g. src/usr.bin/sort/sort.c. > > > > if it's possible to automate this within reasonable bounds, this would > > be better than asking the user to adjust login.conf themselves. > > > > > > Hello, > > lrzip's resource consumption is just brutal. If you try to compress 4 Gb > worth of data it will try, by default, to use 4 GB of RAM or all > available physical memory, whatever is the smallest.
https://github.com/ckolivas/lrzip/blob/master/lrzip.c#L100= So it ought to consider "memory available to the process" based on getrlimit(RLIMIT_DATA, ...) instead of "available physical memory" (Yet) another program that thinks it has a whole machine to itself.. > I personally use lrzip for massive archives (+16 GB) and, judging by > Kolivas' documentation, his idea is for lrzip to be used that way. > > Can we patch the port to do the RLIMIT_DATA trick? Maybe. I'd be > surprised if that was enough for using lrzip as intended, though :-) btw the default datasize limit for a member of "staff" class on amd64 is already 4GB. > I will play a bit with it taking Omar's advice into consideration and > try to come up with a better iteration of this port. > > Cheers! > > -- > OpenPGP Key Fingerprint: > 543F EB89 7FDE 8E33 AFF7 E794 E4AB 4807 58F7 6C76
