Re: xz -9 : Cannot allocate memory
Corinna Vinschen writes: > That's an awful lot of memory, and it it could easily break later mmap's > or thread stack reservations on 32 bit. Given that the default heap is > 384 Megs already, I would prefer if we reserve subsequent heap memory in > chunks of 1 or 2 Megs only, to lower the pressure on the VM. Sounds good, 2MiB should be what Windows uses for "large pages" on processors that have these (in PAE mode or on x86_64). I'm not sure if it still needs nudging to boot w/ PAE enabled, it would use 4MiB large pages when not in PAE mode, but the large pages are not exposed to the application layer on Win2000 and WinXP anyway. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On Fri, Aug 30, 2013 at 09:00:48PM +0200, Corinna Vinschen wrote: >On Aug 30 19:46, Corinna Vinschen wrote: >> On Aug 30 12:58, Achim Gratz wrote: >> > Corinna Vinschen cygwin.com> writes: >> > > Yes, looks normal and expected from what you observed. mmap commits >> > > memory top-down and that was apparently the first free slot big enough >> > > to fullfil the request. The default heap size is 384 Megs and then >> > > there's apparently not enough space anywhere. >> > >> > Sorry if I'm dense, but does this mean that setting the default heap size >> > to >> > zero does in fact mean it's trying to use 374MiB... wait, yes it is: the >> > memory map is the same when I'm setting the initial heap size to 384MiB >> > with >> > peflags. >> > >> > So, even though that rather large heap is essentially unused at the point >> > of >> > failure and there is enough memory free just beyond the heap, the >> > allocation >> > still fails because both the existing heap and the free space are both >> > smaller than what's requested? >> >> Well, good question. You could debug the sbrk function and see why >> it fails to reserve the space. > >I just debugged this and it seems our sbrk implementation has a serious >problem to extend the heap if the new chunk of memory requires to commit >some of the existing heap and to reserve and commit some more space. >It tries to reserve memory using the wrong address and the wrong size. >It also uses a too simple method to commit the memory. I'll apply a fix >shortly. It looked fine to me. The only code change that I did make was to change your coercion of pointers to ptrdiff_t types into (char *), since that is more in keeping with the rest of the code. I had a hard time reading the code with all of the cygheap-> pointers so I changed two functions to be a user_info_heap methods. That, IMO, makes the code faster and more readable. You probably saw that I fixed the USR1/USR2 issue. That was a serious signal bug. I think both of those warrant a new release in fact. cgf -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On Aug 30 17:03, Christopher Faylor wrote: > On Fri, Aug 30, 2013 at 09:00:48PM +0200, Corinna Vinschen wrote: > >On Aug 30 19:46, Corinna Vinschen wrote: > >> On Aug 30 12:58, Achim Gratz wrote: > >> > Corinna Vinschen cygwin.com> writes: > >> > > Yes, looks normal and expected from what you observed. mmap commits > >> > > memory top-down and that was apparently the first free slot big enough > >> > > to fullfil the request. The default heap size is 384 Megs and then > >> > > there's apparently not enough space anywhere. > >> > > >> > Sorry if I'm dense, but does this mean that setting the default heap > >> > size to > >> > zero does in fact mean it's trying to use 374MiB... wait, yes it is: the > >> > memory map is the same when I'm setting the initial heap size to 384MiB > >> > with > >> > peflags. > >> > > >> > So, even though that rather large heap is essentially unused at the > >> > point of > >> > failure and there is enough memory free just beyond the heap, the > >> > allocation > >> > still fails because both the existing heap and the free space are both > >> > smaller than what's requested? > >> > >> Well, good question. You could debug the sbrk function and see why > >> it fails to reserve the space. > > > >I just debugged this and it seems our sbrk implementation has a serious > >problem to extend the heap if the new chunk of memory requires to commit > >some of the existing heap and to reserve and commit some more space. > >It tries to reserve memory using the wrong address and the wrong size. > >It also uses a too simple method to commit the memory. I'll apply a fix > >shortly. > > It looked fine to me. > > The only code change that I did make was to change your coercion of > pointers to ptrdiff_t types into (char *), since that is more in keeping > with the rest of the code. > > I had a hard time reading the code with all of the cygheap-> pointers so > I changed two functions to be a user_info_heap methods. That, IMO, > makes the code faster and more readable. Good idea, that was an overdue change, I guess. What still bugs me in sbrk is the fact that it prefers to reserve subsequent chunks of heap in "chunk"-sized chunks (uh oh). When using the default heap size of 384 Megs, "chunk" and thus each subsequent heap chunk will be 384 Megs as well if the space is still available. That's an awful lot of memory, and it it could easily break later mmap's or thread stack reservations on 32 bit. Given that the default heap is 384 Megs already, I would prefer if we reserve subsequent heap memory in chunks of 1 or 2 Megs only, to lower the pressure on the VM. Sounds good? > You probably saw that I fixed the USR1/USR2 issue. That was a serious signal > bug. I think both of those warrant a new release in fact. I try to do that on Sunday. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgp6BIYaHomFk.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
On Aug 30 19:46, Corinna Vinschen wrote: > On Aug 30 12:58, Achim Gratz wrote: > > Corinna Vinschen cygwin.com> writes: > > > Yes, looks normal and expected from what you observed. mmap commits > > > memory top-down and that was apparently the first free slot big enough > > > to fullfil the request. The default heap size is 384 Megs and then > > > there's apparently not enough space anywhere. > > > > Sorry if I'm dense, but does this mean that setting the default heap size to > > zero does in fact mean it's trying to use 374MiB... wait, yes it is: the > > memory map is the same when I'm setting the initial heap size to 384MiB with > > peflags. > > > > So, even though that rather large heap is essentially unused at the point of > > failure and there is enough memory free just beyond the heap, the allocation > > still fails because both the existing heap and the free space are both > > smaller than what's requested? > > Well, good question. You could debug the sbrk function and see why > it fails to reserve the space. I just debugged this and it seems our sbrk implementation has a serious problem to extend the heap if the new chunk of memory requires to commit some of the existing heap and to reserve and commit some more space. It tries to reserve memory using the wrong address and the wrong size. It also uses a too simple method to commit the memory. I'll apply a fix shortly. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgp4c9lpVWDcm.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
On Aug 30 12:58, Achim Gratz wrote: > Corinna Vinschen cygwin.com> writes: > > Yes, looks normal and expected from what you observed. mmap commits > > memory top-down and that was apparently the first free slot big enough > > to fullfil the request. The default heap size is 384 Megs and then > > there's apparently not enough space anywhere. > > Sorry if I'm dense, but does this mean that setting the default heap size to > zero does in fact mean it's trying to use 374MiB... wait, yes it is: the > memory map is the same when I'm setting the initial heap size to 384MiB with > peflags. > > So, even though that rather large heap is essentially unused at the point of > failure and there is enough memory free just beyond the heap, the allocation > still fails because both the existing heap and the free space are both > smaller than what's requested? Well, good question. You could debug the sbrk function and see why it fails to reserve the space. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpzMuRwwmeR6.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
Corinna Vinschen cygwin.com> writes: > Yes, looks normal and expected from what you observed. mmap commits > memory top-down and that was apparently the first free slot big enough > to fullfil the request. The default heap size is 384 Megs and then > there's apparently not enough space anywhere. Sorry if I'm dense, but does this mean that setting the default heap size to zero does in fact mean it's trying to use 374MiB... wait, yes it is: the memory map is the same when I'm setting the initial heap size to 384MiB with peflags. So, even though that rather large heap is essentially unused at the point of failure and there is enough memory free just beyond the heap, the allocation still fails because both the existing heap and the free space are both smaller than what's requested? > Keep in mind that a > request of 674Mib is a third of the available VM on 32 bit machines. > So, drop the heapsize and go with it, or use the -M option. Will definitely do for my local xz package. Actually, enlarging the initial heap size to more than 674MiB will work as well, there's just these intermediate heap sizes that will fail (and unfortunately the default heap size is in that range). Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On Aug 30 09:26, Achim Gratz wrote: > Achim Gratz nexgo.de> writes: > > > Load xz under GDB, break > > > on "cygwin_exit", run it, and when it hits the breakpoint, observer the > > > memory layout, either in GDB, or by cat'ing /proc/≤xz's pid>/maps. > > > > OK, if it is still reproducible tomorrow I'll have a look. > > I'm not sure I'm looking at the right place, but these seem to be the > relevant differences between the non-working and the working version > (initial heap set to 1MiB) memory maps (I've cut the lines with different > thread ID etc.). > > --- map_0x0.txt 2013-08-30 11:14:33.717797000 +0200 > +++ map_0x1.txt 2013-08-30 11:12:29.171669000 +0200 > @@ -1,40 +1,41 @@ > [...] > -002F-00314000 rw-p : 0 [win heap 0 > default grow] > -00314000-003F ===p 00024000 : 0 [win heap 0 > default grow] > +0030-00324000 rw-p : 0 [win heap 0 > default grow] > +00324000-0040 ===p 00024000 : 0 [win heap 0 > default grow] > [...] > 2000-2009 rw-p : 0 [heap] > -2009-3800 ===p 0009 : 0 [heap] > +2009-2040 ===p 0009 : 0 [heap] > +34C8-58CD rw-p : 0 > 58CD-58CD1000 r--p 76D4:F5BA 2533274790634491 > /usr/bin/cyglzma-5.dll > 58CD1000-58CE8000 r-xp 1000 76D4:F5BA 2533274790634491 > /usr/bin/cyglzma-5.dll Yes, looks normal and expected from what you observed. mmap commits memory top-down and that was apparently the first free slot big enough to fullfil the request. The default heap size is 384 Megs and then there's apparently not enough space anywhere. Keep in mind that a request of 674Mib is a third of the available VM on 32 bit machines. So, drop the heapsize and go with it, or use the -M option. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpGjY1SFcOYd.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
Achim Gratz nexgo.de> writes: > > Load xz under GDB, break > > on "cygwin_exit", run it, and when it hits the breakpoint, observer the > > memory layout, either in GDB, or by cat'ing /proc/≤xz's pid>/maps. > > OK, if it is still reproducible tomorrow I'll have a look. I'm not sure I'm looking at the right place, but these seem to be the relevant differences between the non-working and the working version (initial heap set to 1MiB) memory maps (I've cut the lines with different thread ID etc.). --- map_0x0.txt 2013-08-30 11:14:33.717797000 +0200 +++ map_0x1.txt 2013-08-30 11:12:29.171669000 +0200 @@ -1,40 +1,41 @@ [...] -002F-00314000 rw-p : 0 [win heap 0 default grow] -00314000-003F ===p 00024000 : 0 [win heap 0 default grow] +0030-00324000 rw-p : 0 [win heap 0 default grow] +00324000-0040 ===p 00024000 : 0 [win heap 0 default grow] [...] 2000-2009 rw-p : 0 [heap] -2009-3800 ===p 0009 : 0 [heap] +2009-2040 ===p 0009 : 0 [heap] +34C8-58CD rw-p : 0 58CD-58CD1000 r--p 76D4:F5BA 2533274790634491 /usr/bin/cyglzma-5.dll 58CD1000-58CE8000 r-xp 1000 76D4:F5BA 2533274790634491 /usr/bin/cyglzma-5.dll Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
Charles Wilson writes: >> Right. But is a bit lame that it just fails. Does it have to use >> this big buffer or would it also work with a smaller buffer, if there's >> no memory chunk big enough? > > From what I understand, it will /not/ be able to perform the kinds of > tasks that -9 specifies. E.g. -9 represents a collection of tuning > parameters, including such things as "and use a gigantic dictionary to > store bit sequences that may repeat, and hence be represented by > shortened codes". If you want a -9-sized dictionary, and you can't > get it...xz shouldn't fall back to a -8-sized one. Correct. However I had actually used --best (which is an alias for -9), which probably should tune down when -9 won't work, especially when given a file (just for testing of course) that is not even a single kiB and so it wouldn't make any difference at all which dictionary size is chosen. Normally I'd feed it something of several dozens of MiB, though. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Waldorf MIDI Implementation & additional documentation: http://Synth.Stromeko.net/Downloads.html#WaldorfDocs -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
Corinna Vinschen writes: > I'm not exactly surprised. You're on a 32 bit machine, so you only have > 2 Gigs VM. Probably some DLLs are in the way. I know, also it might be true I've hit a less fortunate meory layout than usual this time. However, I know there's plenty of unused memory before starting that test and it _always_ works whenever the initial heap size is set to anything else than just 0x0. The original discussion was about setting very large initial heaps, but the actual size doesn't seem to matter, which seems, well, odd. > Load xz under GDB, break > on "cygwin_exit", run it, and when it hits the breakpoint, observer the > memory layout, either in GDB, or by cat'ing /proc//maps. OK, if it is still reproducible tomorrow I'll have a look. Regards, Achim. -- +<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+ Samples for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldSamplesExtra -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On 8/29/2013 12:00 PM, Corinna Vinschen wrote: On Aug 29 11:32, Charles Wilson wrote: So...this is NOTABUG, right? Right. But is a bit lame that it just fails. Does it have to use this big buffer or would it also work with a smaller buffer, if there's no memory chunk big enough? From what I understand, it will /not/ be able to perform the kinds of tasks that -9 specifies. E.g. -9 represents a collection of tuning parameters, including such things as "and use a gigantic dictionary to store bit sequences that may repeat, and hence be represented by shortened codes". If you want a -9-sized dictionary, and you can't get it...xz shouldn't fall back to a -8-sized one. -- Chuck -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On Aug 29 11:32, Charles Wilson wrote: > On 8/29/2013 11:11 AM, Corinna Vinschen wrote: > >On Aug 29 14:34, Achim Gratz wrote: > >>Achim Gratz NexGo.DE> writes: > >>>With the initial heap size set to 1MB, the same allocations for mmap work > >>>just fine. The mmap length used by xz is the same for files of all sizes. > >> > >>The large mmap is for the scratchpad memory of xz, which can be limited via > >>the -M option. Currently it fails when this gets larger than 616MiB > >>(standard for -9 is 674Mib). > > > >I'm not exactly surprised. You're on a 32 bit machine, so you only have > >2 Gigs VM. Probably some DLLs are in the way. Load xz under GDB, break > >on "cygwin_exit", run it, and when it hits the breakpoint, observer the > >memory layout, either in GDB, or by cat'ing /proc//maps. > >Observing the memory layout helps a lot to understand why there's some > >memory problem, especially if the application tries to get a big buffer > >space in a single chunk. > > So...this is NOTABUG, right? Right. But is a bit lame that it just fails. Does it have to use this big buffer or would it also work with a smaller buffer, if there's no memory chunk big enough? Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpDjG3EYBh6i.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
On 8/29/2013 11:11 AM, Corinna Vinschen wrote: On Aug 29 14:34, Achim Gratz wrote: Achim Gratz NexGo.DE> writes: With the initial heap size set to 1MB, the same allocations for mmap work just fine. The mmap length used by xz is the same for files of all sizes. The large mmap is for the scratchpad memory of xz, which can be limited via the -M option. Currently it fails when this gets larger than 616MiB (standard for -9 is 674Mib). I'm not exactly surprised. You're on a 32 bit machine, so you only have 2 Gigs VM. Probably some DLLs are in the way. Load xz under GDB, break on "cygwin_exit", run it, and when it hits the breakpoint, observer the memory layout, either in GDB, or by cat'ing /proc//maps. Observing the memory layout helps a lot to understand why there's some memory problem, especially if the application tries to get a big buffer space in a single chunk. So...this is NOTABUG, right? If you don't understand the options and their implications, and end up (as with -9) trying to allocate a giant hunk of memory unsupported by the memory model (e.g. 2GB VM), it fails. I think that's preferable to falling back to -8 or -7 or something. -- Chuck -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On Aug 29 14:34, Achim Gratz wrote: > Achim Gratz NexGo.DE> writes: > > With the initial heap size set to 1MB, the same allocations for mmap work > > just fine. The mmap length used by xz is the same for files of all sizes. > > The large mmap is for the scratchpad memory of xz, which can be limited via > the -M option. Currently it fails when this gets larger than 616MiB > (standard for -9 is 674Mib). I'm not exactly surprised. You're on a 32 bit machine, so you only have 2 Gigs VM. Probably some DLLs are in the way. Load xz under GDB, break on "cygwin_exit", run it, and when it hits the breakpoint, observer the memory layout, either in GDB, or by cat'ing /proc//maps. Observing the memory layout helps a lot to understand why there's some memory problem, especially if the application tries to get a big buffer space in a single chunk. Corinna -- Corinna Vinschen Please, send mails regarding Cygwin to Cygwin Maintainer cygwin AT cygwin DOT com Red Hat pgpTNcnD6mh8C.pgp Description: PGP signature
Re: xz -9 : Cannot allocate memory
Achim Gratz NexGo.DE> writes: > With the initial heap size set to 1MB, the same allocations for mmap work > just fine. The mmap length used by xz is the same for files of all sizes. The large mmap is for the scratchpad memory of xz, which can be limited via the -M option. Currently it fails when this gets larger than 616MiB (standard for -9 is 674Mib). Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
David Stacey tiscali.co.uk> writes: > I am trying to use 'xz -9' to compress a file, but the programme exits > with the error message 'Cannot allocate memory'. Here's what I tried: > > $ echo Hello World > compress_me.txt > > $ xz -9 compress_me.txt > xz: compress_me.txt: Cannot allocate memory [...] > And this worked. Is this the correct way to fix the problem? If so, > please could we increment the heap size for xz in a post install script? I'm using a self-compiled xz-5.0.5 that has started to exhibit that very problem after updating to Cygwin 1.7.24. Setting the initial heap size to _anything_ but 0x0 will make the error go away. I've run strace on it and the error propagates from mmap64 apparently: 81 64830 [main] xz 17764 fstat64: 0 = fstat(3, 0x4141A0) 44 64874 [main] xz 17764 set_signal_mask: setmask 21807003, newmask 0, mask_bits 21807003 34 64908 [main] xz 17764 pthread_sigmask: 0 = pthread_sigmask(2, 0x4144A8, 0x0) 46 64954 [main] xz 17764 read: read(3, 0x410160, 8192) blocking 1199 66153 [main] xz 17764 fhandler_base::read: returning 8192, binary mode 45 66198 [main] xz 17764 read: 8192 = read(3, 0x410160, 8192) 355 66553 [main] xz 17764 mmap64: addr 0x0, len 101253120, prot 0x3, flags 0x22, fd -1, off 0x0 402 66955 [main] xz 17764 mmap64: 0x7966 = mmap() 50 67005 [main] xz 17764 mmap64: addr 0x0, len 604307456, prot 0x3, flags 0x22, fd -1, off 0x0 41 67046 [main] xz 17764 seterrno_from_win_error: /home/c.../winsup/cygwin/mmap.cc:1641 windows error 8 62 67108 [main] xz 17764 geterrno_from_win_error: windows error 8 == errno 12 23 67131 [main] xz 17764 mmap64: 0x = mmap() 29 67160 [main] xz 17764 __set_errno: void* sbrk(ptrdiff_t):303 setting errno 12 37 67197 [main] xz 17764 mmap64: addr 0x0, len 604307456, prot 0x3, flags 0x22, fd -1, off 0x0 33 67230 [main] xz 17764 seterrno_from_win_error: /home/c.../winsup/cygwin/mmap.cc:1641 windows error 8 23 67253 [main] xz 17764 geterrno_from_win_error: windows error 8 == errno 12 25 67278 [main] xz 17764 mmap64: 0x = mmap() With the initial heap size set to 1MB, the same allocations for mmap work just fine. The mmap length used by xz is the same for files of all sizes. Regards, Achim. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On 15/11/12 06:30, JonY wrote: On 11/15/2012 13:37, Peter Rosin wrote: On 2012-11-15 05:40, marco atzeri wrote: On 11/14/2012 11:09 PM, David Stacey wrote: I am trying to use 'xz -9' to compress a file, but the programme exits with the error message 'Cannot allocate memory'. Here's what I tried: $ echo Hello World > compress_me.txt $ xz -9 compress_me.txt xz: compress_me.txt: Cannot allocate memory $ xz --version xz (XZ Utils) 5.0.2 liblzma 5.0.2 Having read http://cygwin.com/cygwin-ug-net/setup-maxmem.html I tried the following: $ peflags --cygwin-heap /usr/bin/xz.exe /usr/bin/xz.exe: initial Cygwin heap size: 0 (0x0) MB $ peflags --cygwin-heap=1024 /usr/bin/xz.exe /usr/bin/xz.exe: initial Cygwin heap size: 1024 (0x400) MB $ xz -9 compress_me.txt And this worked. Is this the correct way to fix the problem? If so, please could we increment the heap size for xz in a post install script? a lot of programs have "Cygwin heap size: 0 (0x0) MB" but they work anyway so the root cause is somewhere else. xz is notorious for its big up-front allocations, at least with some of the more aggressive options in effect. The -9 option works for me without modifying Cygwin heap. Thank you all for your help. I have repeated the above test using /usr/i686-pc-mingw32/sys-root/mingw/bin/xz -9 and this worked OK. This is the same version of xz (5.0.2), but compiled up using mingw (and therefore presumably not using cygwin1.dll). If it helps in reproducing this problem, I am /still/ using Windows XP Pro SP3. Many thanks, Dave. -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On 11/15/2012 13:37, Peter Rosin wrote: > On 2012-11-15 05:40, marco atzeri wrote: >> On 11/14/2012 11:09 PM, David Stacey wrote: >>> I am trying to use 'xz -9' to compress a file, but the programme exits >>> with the error message 'Cannot allocate memory'. Here's what I tried: >>> >>> >>> $ echo Hello World > compress_me.txt >>> >>> $ xz -9 compress_me.txt >>> xz: compress_me.txt: Cannot allocate memory >>> >>> $ xz --version >>> xz (XZ Utils) 5.0.2 >>> liblzma 5.0.2 >>> >>> >>> Having read http://cygwin.com/cygwin-ug-net/setup-maxmem.html I tried >>> the following: >>> >>> >>> $ peflags --cygwin-heap /usr/bin/xz.exe >>> /usr/bin/xz.exe: initial Cygwin heap size: 0 (0x0) MB >>> >>> $ peflags --cygwin-heap=1024 /usr/bin/xz.exe >>> /usr/bin/xz.exe: initial Cygwin heap size: 1024 (0x400) MB >>> >>> $ xz -9 compress_me.txt >>> >>> >>> And this worked. Is this the correct way to fix the problem? If so, >>> please could we increment the heap size for xz in a post install script? >> >> a lot of programs have "Cygwin heap size: 0 (0x0) MB" >> but they work anyway >> >> so the root cause is somewhere else. > > xz is notorious for its big up-front allocations, at least with some of > the more aggressive options in effect. > The -9 option works for me without modifying Cygwin heap. signature.asc Description: OpenPGP digital signature
Re: xz -9 : Cannot allocate memory
On 2012-11-15 05:40, marco atzeri wrote: > On 11/14/2012 11:09 PM, David Stacey wrote: >> I am trying to use 'xz -9' to compress a file, but the programme exits >> with the error message 'Cannot allocate memory'. Here's what I tried: >> >> >> $ echo Hello World > compress_me.txt >> >> $ xz -9 compress_me.txt >> xz: compress_me.txt: Cannot allocate memory >> >> $ xz --version >> xz (XZ Utils) 5.0.2 >> liblzma 5.0.2 >> >> >> Having read http://cygwin.com/cygwin-ug-net/setup-maxmem.html I tried >> the following: >> >> >> $ peflags --cygwin-heap /usr/bin/xz.exe >> /usr/bin/xz.exe: initial Cygwin heap size: 0 (0x0) MB >> >> $ peflags --cygwin-heap=1024 /usr/bin/xz.exe >> /usr/bin/xz.exe: initial Cygwin heap size: 1024 (0x400) MB >> >> $ xz -9 compress_me.txt >> >> >> And this worked. Is this the correct way to fix the problem? If so, >> please could we increment the heap size for xz in a post install script? > > a lot of programs have "Cygwin heap size: 0 (0x0) MB" > but they work anyway > > so the root cause is somewhere else. xz is notorious for its big up-front allocations, at least with some of the more aggressive options in effect. Cheers, Peter -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Re: xz -9 : Cannot allocate memory
On 11/14/2012 11:09 PM, David Stacey wrote: I am trying to use 'xz -9' to compress a file, but the programme exits with the error message 'Cannot allocate memory'. Here's what I tried: $ echo Hello World > compress_me.txt $ xz -9 compress_me.txt xz: compress_me.txt: Cannot allocate memory $ xz --version xz (XZ Utils) 5.0.2 liblzma 5.0.2 Having read http://cygwin.com/cygwin-ug-net/setup-maxmem.html I tried the following: $ peflags --cygwin-heap /usr/bin/xz.exe /usr/bin/xz.exe: initial Cygwin heap size: 0 (0x0) MB $ peflags --cygwin-heap=1024 /usr/bin/xz.exe /usr/bin/xz.exe: initial Cygwin heap size: 1024 (0x400) MB $ xz -9 compress_me.txt And this worked. Is this the correct way to fix the problem? If so, please could we increment the heap size for xz in a post install script? a lot of programs have "Cygwin heap size: 0 (0x0) MB" but they work anyway so the root cause is somewhere else. Dave. please look at Problem reports: http://cygwin.com/problems.html specially the "Run cygcheck -s -v -r > cygcheck.out and include that file as an attachment in your report" Marco -- Problem reports: http://cygwin.com/problems.html FAQ: http://cygwin.com/faq/ Documentation: http://cygwin.com/docs.html Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
