FreeBSD 7 buildworld error

[Derek Taylor]

I csup'ed to RELENG_7_0 (from RELENG_6_3) yesterday and have been   
having problems building world.  After about 10 minutes of building,
I get the error: cc1: out of memory allocating 97582896 bytes

I have put the full output at
.   

Someone suggested that I might have a ulimit problem, but that looks
ok to me:
$ ulimit -a
core file size  (blocks, -c) unlimited
data seg size   (kbytes, -d) 524288
file size   (blocks, -f) unlimited 
max locked memory   (kbytes, -l) unlimited
max memory size (kbytes, -m) unlimited
open files  (-n) 11095
pipe size(512 bytes, -p) 1   
stack size  (kbytes, -s) 65536
cpu time   (seconds, -t) unlimited
max user processes  (-u) 5547
virtual memory  (kbytes, -v) unlimited

Any ideas?  I can provide more information at request.  

-Derek.  



___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Alfred Perlstein]

* Derek Taylor <[EMAIL PROTECTED]> [080229 18:33] wrote:
> I csup'ed to RELENG_7_0 (from RELENG_6_3) yesterday and have been   
> having problems building world.  After about 10 minutes of building,
> I get the error: cc1: out of memory allocating 97582896 bytes
> 
> I have put the full output at
> .   
> 
> Someone suggested that I might have a ulimit problem, but that looks
> ok to me:
> $ ulimit -a
> core file size  (blocks, -c) unlimited
> data seg size   (kbytes, -d) 524288
> file size   (blocks, -f) unlimited 
> max locked memory   (kbytes, -l) unlimited
> max memory size (kbytes, -m) unlimited
> open files  (-n) 11095
> pipe size(512 bytes, -p) 1   
> stack size  (kbytes, -s) 65536
> cpu time   (seconds, -t) unlimited
> max user processes  (-u) 5547
> virtual memory  (kbytes, -v) unlimited
> 
> Any ideas?  I can provide more information at request.  

Object directory clean?

-- 
- Alfred Perlstein
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Derek Taylor]

On Fri, Feb 29, 2008 09:50 PM Sean <[EMAIL PROTECTED]> wrote:
>
>Check /etc/make.conf for CFLAGS, and if present remove it.

This fixed the problem.

Thank you.

-Derek.

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Joshua Coombs]

Derek Taylor wrote:

On Fri, Feb 29, 2008 09:50 PM Sean <[EMAIL PROTECTED]> wrote:

Check /etc/make.conf for CFLAGS, and if present remove it.


This fixed the problem.

Thank you.

-Derek.


I can confirm a failure in the same spot.  What concerns me is in both 
my failure, and Derek's, the malloc is failing well below what limit 
says should be allowed.


bin/cc/cc_int/../cc_tools 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/include 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libcpp/include 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libdecnumber 
 -I/usr/obj/usr/src/tmp/legacy/usr/include -c ../cc_tools/insn-attrtab.c


cc1: out of memory allocating 136475392 bytes
*** Error code 1

Stop in /usr/src/gnu/usr.bin/cc/cc_int.
*** Error code 1

cyrix-dlc# limit
cputime  unlimited
filesize unlimited
datasize 524288 kbytes
stacksize65536 kbytes
coredumpsize unlimited
memoryuseunlimited
vmemoryuse   unlimited
descriptors  957
memorylocked unlimited
maxproc  478
sbsize   unlimited

cc1 was only trying to request 130MB, my datasize is 512MB, why did it fail?

Not to mention the additional question of why is cc1 ballooning up so 
badly on that file, when leading up to it it's using on average 12MB per 
instance?  Prior versions of FreeBSD never took anywhere NEAR this much 
RAM to build.


(I also don't like the 'remove all CFLAGS' "fix" suggestion, something 
is broken if a buildworld can't complete on a stock kernel with sane 
CFLAGS.  In my case, I run -Os -pipe, am I now to understand that any 
CFLAGS setting is inappropriate?)


Joshua Coombs

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Jason Evans]

Joshua Coombs wrote:

Derek Taylor wrote:

On Fri, Feb 29, 2008 09:50 PM Sean <[EMAIL PROTECTED]> wrote:

Check /etc/make.conf for CFLAGS, and if present remove it.


This fixed the problem.

Thank you.

-Derek.


I can confirm a failure in the same spot.  What concerns me is in both 
my failure, and Derek's, the malloc is failing well below what limit 
says should be allowed.


bin/cc/cc_int/../cc_tools 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/include 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libcpp/include 
-I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libdecnumber 
 -I/usr/obj/usr/src/tmp/legacy/usr/include -c ../cc_tools/insn-attrtab.c


cc1: out of memory allocating 136475392 bytes
*** Error code 1

Stop in /usr/src/gnu/usr.bin/cc/cc_int.
*** Error code 1

cyrix-dlc# limit
cputime  unlimited
filesize unlimited
datasize 524288 kbytes
stacksize65536 kbytes
coredumpsize unlimited
memoryuseunlimited
vmemoryuse   unlimited
descriptors  957
memorylocked unlimited
maxproc  478
sbsize   unlimited

cc1 was only trying to request 130MB, my datasize is 512MB, why did it 
fail?


It looks to me like gcc is trying to allocate a single 130MiB object, 
but you don't say anything about how much memory is already in use.  It 
may well be that there are no remaining places in the memory map to 
place such a large object.


Jason
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Joshua Coombs]

Jason Evans wrote:
cc1 was only trying to request 130MB, my datasize is 512MB, why did it 
fail?


It looks to me like gcc is trying to allocate a single 130MiB object, 
but you don't say anything about how much memory is already in use.  It 
may well be that there are no remaining places in the memory map to 
place such a large object.


Jason


The machine is a pretty lightly loaded, but low physical mem box.  64mb 
ram, 384mb of swap normally, with an additional 2gb of swap to a file 
added for my buildworld attempt.


Mem: 20M Active, 5604K Inact, 27M Wired, 2876K Cache, 14M Buf, 3064K Free
Swap: 2432M Total, 21M Used, 2411M Free

15901 root1 1150 12640K 11628K RUN  2:22 86.96% cc1

Thats while building tree-ssa-ccp.c.  How do I test out your theory? 
(Short of watching top while doing the build.)


Josh C

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Erik Trulsson]

On Fri, Mar 07, 2008 at 08:54:53AM -0800, Jason Evans wrote:
> Joshua Coombs wrote:
>> Derek Taylor wrote:
>>> On Fri, Feb 29, 2008 09:50 PM Sean <[EMAIL PROTECTED]> wrote:
 Check /etc/make.conf for CFLAGS, and if present remove it.
>>> 
>>> This fixed the problem.
>>> 
>>> Thank you.
>>> 
>>> -Derek.
>> 
>> I can confirm a failure in the same spot.  What concerns me is in both my 
>> failure, and Derek's, the malloc is failing well below what limit says 
>> should be allowed.
>> 
>> bin/cc/cc_int/../cc_tools 
>> -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc 
>> -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcc/config 
>> -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/include 
>> -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libcpp/include 
>> -I/usr/src/gnu/usr.bin/cc/cc_int/../../../../contrib/gcclibs/libdecnumber  
>> -I/usr/obj/usr/src/tmp/legacy/usr/include -c ../cc_tools/insn-attrtab.c
>> 
>> cc1: out of memory allocating 136475392 bytes
>> *** Error code 1
>> 
>> Stop in /usr/src/gnu/usr.bin/cc/cc_int.
>> *** Error code 1
>> 
>> cyrix-dlc# limit
>> cputime  unlimited
>> filesize unlimited
>> datasize 524288 kbytes
>> stacksize65536 kbytes
>> coredumpsize unlimited
>> memoryuseunlimited
>> vmemoryuse   unlimited
>> descriptors  957
>> memorylocked unlimited
>> maxproc  478
>> sbsize   unlimited
>> 
>> cc1 was only trying to request 130MB, my datasize is 512MB, why did it 
>> fail?
> 
> It looks to me like gcc is trying to allocate a single 130MiB object, but 
> you don't say anything about how much memory is already in use.  It may 
> well be that there are no remaining places in the memory map to place such 
> a large object.

And that particular file (insn-attrtab.c, which is a machine-generated part
of gcc itself,) is known to require much more memory to compile than most
other source files.
If gcc runs out of memory anywhere during a buildworld it is most likely
when compiling that particular file.



-- 

Erik Trulsson
[EMAIL PROTECTED]
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Joshua Coombs]

As a follow up:

I setup a fresh 7.0 install in a VM and played with it.  Using my 
make.conf, I showed cc1 using 130MB when compiling insn-attrtab.c.  I 
tweaked the VM conf down to 32MB of RAM and redid the compile, and other 
than taking forever due to swapping, it again churned past 
insn-attrtab.c using 130MB successfully.


My next test is going to be simulating the swap setup I've got on my 
386.  Rather than one 384MB swap partition, I have three 128MB swap 
partitions, one per drive.  My theory was I'd see some small boost when 
swapping by spreading the work load.  Turns out it doesn't help, I 
bottleneck on the ISA bus, transferring 1.5MB/sec max no matter how I 
spread the work across the drives.  When I re-partition I'll just pick 
one drive to hold all swap.


That said, given how I butted up against a limit suspiciously close to 
the size of my swap partitions, rather than the 512MB data size limit 
reports, thats my next guess as to the cause, swap isn't allowing single 
processes to page into multiple swap pools?


Josh C

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"

Re: FreeBSD 7 buildworld error

[Joshua Coombs]

Joshua Coombs wrote:

As a follow up:

I setup a fresh 7.0 install in a VM and played with it.  Using my 
make.conf, I showed cc1 using 130MB when compiling insn-attrtab.c.  I 
tweaked the VM conf down to 32MB of RAM and redid the compile, and other 
than taking forever due to swapping, it again churned past 
insn-attrtab.c using 130MB successfully.


My next test is going to be simulating the swap setup I've got on my 
386.  Rather than one 384MB swap partition, I have three 128MB swap 
partitions, one per drive.  My theory was I'd see some small boost when 
swapping by spreading the work load.  Turns out it doesn't help, I 
bottleneck on the ISA bus, transferring 1.5MB/sec max no matter how I 
spread the work across the drives.  When I re-partition I'll just pick 
one drive to hold all swap.


That said, given how I butted up against a limit suspiciously close to 
the size of my swap partitions, rather than the 512MB data size limit 
reports, thats my next guess as to the cause, swap isn't allowing single 
processes to page into multiple swap pools?


Josh C


I can now confirm, using two 128MB swap partitions, plus a third 2GB 
partition, with 32MB of ram configured results in buildworld's failing, 
just using a 2GB swap partition works.


Josh C

___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"