Re: [Caml-list] memory usage
On Mon, Jan 12, 2009 at 03:41:46PM +0800, John Lepikhin wrote: Please, give me some start point to find the roots of the problem. Starting point should be to call this periodically: Gc.compact (); let stat = Gc.stat () in let live_words = stat.Gc.live_words in eprintf live words %d\n%! live_words; which will tell you how many words (ie 4 or 8 byte chunks) are reachable according to the garbage collector. If this number is going up, then somewhere you are holding a pointer to some object that you didn't expect to be live. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Starting point should be to call this periodically: Gc.compact (); let stat = Gc.stat () in let live_words = stat.Gc.live_words in eprintf live words %d\n%! live_words; which will tell you how many words (ie 4 or 8 byte chunks) are reachable according to the garbage collector. Richard, here is result (statistics was saved every 10 seconds): live words - RSS: 186980 - 12380KB -- after first 10 seconds of work 154156 - 18232KB 153923 - 19648KB ... after 10 minutes of work: 203842 - 33436KB 170559 - 33528KB 187018 - 33664KB 71626 - 33592KB Sometimes live words drops down to 40.000. But RSS always stay near 30-50MB. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Please, give me some start point to find the roots of the problem. I assume you're running on Linux. I've got this issue on FreeBSD box. On Linux test box process grows only up to 2MB (but there is no production data and clients). Here is partial pmap output: Address KbytesRSS Shared Priv Mode Mapped File ... 408AA000932548932 - r-x /lib/libc.so.7 40993000 4 4 4 - r-x /lib/libc.so.7 40994000 1020 0 1020 - r-x /lib/libc.so.7 40A93000116116 -116 rw- /lib/libc.so.7 40AB 92 44 - 92 rw- [ anon ] 40B0 1024904 - 1024 rw- [ anon ] 40C0 34816 19652 - 34816 rw- [ anon ] 42E0 1024200 - 1024 rw- [ anon ] 42F0 2048400 - 2048 rw- [ anon ] 7B9BE000128 20 -128 rw- [ anon ] 7BBBF000128 20 -128 rw- [ anon ] ... -- -- -- -- Total Kb 48552 23736 4352 44528 Most of memory is allocated here: 40C0 34816 19652 - 34816 rw- [ anon ] It gets bigger and bigger: 40C0 34816 20660 - 34816 rw- [ anon ] 40C0 34816 21216 - 34816 rw- [ anon ] One other thing to check is to ensure that threads really are being killed at the times you expect. (Check /proc/pid/status | grep Threads, or use gdb -p to attach to the running process then info thr.) I checked it. Only valid threads run. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Each thread is killed after work is done How do you kill the threads? I hope this is just a figure of speech for I do an orderly shutdown of each thread after work is done. Well, that was consequence of my bad English :-) Threads finish their work and exit. I also made a simple wrapper to Thread.create to be sure that all work inside threads is done: module MyThread = let create f p = let dowork _ = (* log thread creation *) f p; (* log thread shutdown *) in Thread.create dowork () end ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] memory usage
Hello, I have developed an application and have issue with memory usage. Application was designed for work as server, 24x7. After several hours of work, RSS memory grows up to 40-50MB. Each thread must not use more than 100-200KB of memory (maximum 20 threads are run at the same time). GC debug information: Growing heap to 3840k bytes Growing heap to 4320k bytes Shrinking heap to 3840k bytes Growing heap to 4320k bytes Growing heap to 4800k bytes Growing heap to 5280k bytes Shrinking heap to 4800k bytes Shrinking heap to 4320k bytes Shrinking heap to 3840k bytes Growing heap to 4320k bytes Growing heap to 4800k bytes Growing heap to 5280k bytes Shrinking heap to 4800k bytes Shrinking heap to 4320k bytes Shrinking heap to 3840k bytes Shrinking heap to 3360k bytes Growing heap to 3840k bytes Growing heap to 4320k bytes Growing heap to 4800k bytes Growing heap to 960k bytes Shrinking heap to 4320k bytes Shrinking heap to 3840k bytes Shrinking heap to 3360k bytes Shrinking heap to 2880k bytes Growing heap to 3360k bytes Shrinking heap to 2880k bytes Shrinking heap to 2400k bytes Growing heap to 2880k bytes Growing heap to 3360k bytes Growing heap to 3840k bytes Shrinking heap to 3360k bytes Shrinking heap to 2880k bytes Growing heap to 3360k bytes Shrinking heap to 2880k bytes As you can see, heap size never gets bigger 5-6MB. I made core dump of process. 90% of memory was filled with values which are created inside threads and never been copied outside of them. Each thread is killed after work is done, I am absolutely sure in it; all unused file descriptors are closed. Playing with Gc.set has not brought notable results. Please, give me some start point to find the roots of the problem. Sorry for my English. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Good news, I just tested the patch and it works great with my application! I just had to modify the module random since a call to (Random.int max_int) may raise and exception (it is made for 32 bits integers). So I guess that modification should be included in the patch. Thanks a lot Andres. Jean On Wed, Jul 16, 2008 at 12:27 PM, Jean Krivine [EMAIL PROTECTED] wrote: Great thanks! J On Wed, Jul 16, 2008 at 10:16 AM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 15, 2008, at 3:38 PM, Jean Krivine wrote: I'd be glad to try the patch if you could post it somewhere! I have posted it in: http://research.amnh.org/~avaron/ocaml/ best, Andres J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
On Jul 16, 2008, at 2:07 PM, Jean Krivine wrote: Good news, I just tested the patch and it works great with my application! I just had to modify the module random since a call to (Random.int max_int) may raise and exception (it is made for 32 bits integers). So I guess that modification should be included in the patch. I don't think that's a good idea. You have to use Random.int64 to get a 64 bit random integer. The Random.int function will return an integer between 0 and 2^30. Check the Random module documentation here: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Random.html I wouldn't play with a random number generator unless I know exactly what I'm doing. Your results depend on it! (well, your messed-up-by- andres compiler could already have issues ... :-(, for what I use it I can verify the result with a 32 bit binary or a 64 bit linux binary, if you can, then do the same!). Andres Thanks a lot Andres. Jean On Wed, Jul 16, 2008 at 12:27 PM, Jean Krivine [EMAIL PROTECTED] wrote: Great thanks! J On Wed, Jul 16, 2008 at 10:16 AM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 15, 2008, at 3:38 PM, Jean Krivine wrote: I'd be glad to try the patch if you could post it somewhere! I have posted it in: http://research.amnh.org/~avaron/ocaml/ best, Andres J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/ experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
I agree. I should use Int64 instead of just int, but I still think that the application (Random.int max_int) should not be exception prone. Since max_int is architecture dependent, then so should be Random.int no? But you point is well taken. Thanks again J On Wed, Jul 16, 2008 at 2:44 PM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 16, 2008, at 2:07 PM, Jean Krivine wrote: Good news, I just tested the patch and it works great with my application! I just had to modify the module random since a call to (Random.int max_int) may raise and exception (it is made for 32 bits integers). So I guess that modification should be included in the patch. I don't think that's a good idea. You have to use Random.int64 to get a 64 bit random integer. The Random.int function will return an integer between 0 and 2^30. Check the Random module documentation here: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Random.html I wouldn't play with a random number generator unless I know exactly what I'm doing. Your results depend on it! (well, your messed-up-by-andres compiler could already have issues ... :-(, for what I use it I can verify the result with a 32 bit binary or a 64 bit linux binary, if you can, then do the same!). Andres Thanks a lot Andres. Jean On Wed, Jul 16, 2008 at 12:27 PM, Jean Krivine [EMAIL PROTECTED] wrote: Great thanks! J On Wed, Jul 16, 2008 at 10:16 AM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 15, 2008, at 3:38 PM, Jean Krivine wrote: I'd be glad to try the patch if you could post it somewhere! I have posted it in: http://research.amnh.org/~avaron/ocaml/ best, Andres J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Re: [Caml-list] memory usage
Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
I'd be glad to try the patch if you could post it somewhere! J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] memory usage
Dear list members, I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Thank you very much! Jean ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
It is hard to tell without any more informations but sometimes the garbage collector needs some gentle proding: OCaml handles it's own memory but can be a bad citizen when it comes to making room for others. Unfortunately ocaml also has a bit of a double personality: it doesn't know much about resources used in external libraries or even in some of its own library (e.g. on a 32 bits machine running out of addressable space because of Bigarray.map_file is not unheard of). If this is your problem, you can either sprinkle your source code with calls to Gc.major or tweak it using Gc.set. Till On Fri, Jul 11, 2008 at 8:49 PM, Jean Krivine [EMAIL PROTECTED] wrote: Dear list members, I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Thank you very much! Jean ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- http://till-varoquaux.blogspot.com/ ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
