[Caml-list] Garbage collection and profiling
Dear list Does anyone knows a simple way to evaluate the time spent by an ocaml program in GCing? I guess I can simply disable the GC and compare execution time but I was wondering whether there is no direct way to do it. Thanks! ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Garbage collection and profiling
For some strange reasons when I do so, I don't have time% but only call numbers in the profile. Is there something I missed? J On Thu, Jun 4, 2009 at 1:29 PM, David MENTRE dmen...@linux-france.org wrote: Hello, On Thu, Jun 4, 2009 at 19:03, Jean Krivine jean.kriv...@gmail.com wrote: Does anyone knows a simple way to evaluate the time spent by an ocaml program in GCing? Activate profiling in native code version and sum the time spent in the GC entry points in gprof call graph output? Sincerely yours, david ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Manipulating xml files withing ocaml
Just a quick question, I would like to open and access easily an xml file within my ocaml program. Does someone have a small library that would help me doing that? All the things I found for now are libraries that help me construct xml and check for consistency which is not what I want. Thanks! Jean ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Manipulating xml files withing ocaml
Superb that seems to be what I needed thanks! J On Sun, Nov 23, 2008 at 10:18 AM, Romain Beauxis [EMAIL PROTECTED] wrote: Hi Jean ! On Sun, 23 Nov 2008 10:13:27 -0700, Jean Krivine [EMAIL PROTECTED] wrote: Just a quick question, I would like to open and access easily an xml file within my ocaml program. Does someone have a small library that would help me doing that? All the things I found for now are libraries that help me construct xml and check for consistency which is not what I want. If the xml you intend to parse is not too broken (like html for instance), you can try to use xml-light, whose APIis very simple. The link: http://tech.motion-twin.com/xmllight.html Romain ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: Re : [Caml-list] Manipulating xml files withing ocaml
In my case I just want to parse a small xml file and I don't need to check for consistency so I think xml-light is fine enough. Thank you for the references though J On Sun, Nov 23, 2008 at 10:54 AM, Dario Teixeira [EMAIL PROTECTED] wrote: PXP may be a better choice. It's harder at first, but not much and could be worth the five additionnal minutes required to learn it. It handles unicode and is still maintained (among others). Indeed. If you find yourself routinely having to deal with XML, and if you need more advanced features such as validating very complex DTDs, than learning PXP is well worth it (Xml-light will choke on all but the simplest DTDs). If you are doing heavy manipulation of XML and wished for a more native support in the language, then take a look at Cduce/Ocamlduce. Finally, if you just need to parse a simple XML file and this is a one-off event, then Xml-light is indeed the simplest option. Cheers, Dario Teixeira ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Difference between let rec and just let?
Each time you use let, you can also use let rec, but you have to use the rec (for recursive) if you want to declare a recursive function as in: let rec f x = if x0 then 0 else x * f(x-1) On Wed, Aug 13, 2008 at 8:49 AM, circ ular [EMAIL PROTECTED] wrote: what is the difference between let rec and just let? what does rec stand for? are the following defintions exactly the same? at least they seem to give the same results... # let rec cube x = x*x*x;; val cube : int - int = fun # cube 12;; - : int = 1728 # let cubex x = x*x*x;; val cubex : int - int = fun # cubex 12;; - : int = 1728 # ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Manually triggering garbage collection
OK great I' ll try, For the moment I just set a Gc alarm that detects whether memory usage is above a certain limit and if so, sets the overhead to 0, which stops completely the memory leak. Do you think that would improve to increase the size of the major heap? Also, do you know how often the alarm is tested? is it each time a major collection is performed? Thanks a lot J On Tue, Jul 29, 2008 at 8:53 AM, Damien Doligez [EMAIL PROTECTED] wrote: Hello Jean, On 2008-07-26, at 21:15, Jean Krivine wrote: I am running a memory intensive stochastic simulator written in ocaml. After initialization of the data structure (which eats up a lot of memory but that's normal) I observe a memory leak during the simulation which should not be there. I noticed that if I run Gc.major() every n computation events after initialization (I can make n vary), then there is no more memory leak (the memory the process is using is constant). So my question is the following: Is there a rational way to detect I should call for Gc.Major()? (for the moment I am triggering it every n events which is arbitrary) This might be a fragmentation problem, but if calling Gc.major() fixes it, I guess it's not a true leak (i.e. the memory would stabilize at some point). At any rate, you might want to try the CVS head version (3.11+dev): it uses first-fit allocation in order to fight fragmentation, and I'd like to know how well it does in the wild. -- Damien ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Manually triggering garbage collection
Well if I don't set the overhead to 0, there is still a slight innactive memory accumulation. However, when the memory goes back to a reasonable level, I set the overhead back to 80 (which is the default value) using another alarm. The result of this is that inactive memory wobbles continuously during the execution (it accumulates at overhead 80 and decreases at overhead 0). J On Tue, Jul 29, 2008 at 9:57 AM, Damien Doligez [EMAIL PROTECTED] wrote: On 2008-07-29, at 15:39, Jean Krivine wrote: OK great I' ll try, For the moment I just set a Gc alarm that detects whether memory usage is above a certain limit and if so, sets the overhead to 0, which stops completely the memory leak. Setting the overhead to 0 seems a bit overkill. You might want to do a Gc.compact() at this point. Or you might just need to change the max_overhead parameter: the lower it is, the more often the GC will compact the heap. Do you think that would improve to increase the size of the major heap? Yes, sometimes it helps with fragmentation. Also, increasing the heap_increment may help. Also, do you know how often the alarm is tested? is it each time a major collection is performed? The GC alarms are triggered at the end of each major collection. -- Damien ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Manually triggering garbage collection
Dear list, I am running a memory intensive stochastic simulator written in ocaml. After initialization of the data structure (which eats up a lot of memory but that's normal) I observe a memory leak during the simulation which should not be there. I noticed that if I run Gc.major() every n computation events after initialization (I can make n vary), then there is no more memory leak (the memory the process is using is constant). So my question is the following: Is there a rational way to detect I should call for Gc.Major()? (for the moment I am triggering it every n events which is arbitrary) Thank you very much Jean ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
Good news, I just tested the patch and it works great with my application! I just had to modify the module random since a call to (Random.int max_int) may raise and exception (it is made for 32 bits integers). So I guess that modification should be included in the patch. Thanks a lot Andres. Jean On Wed, Jul 16, 2008 at 12:27 PM, Jean Krivine [EMAIL PROTECTED] wrote: Great thanks! J On Wed, Jul 16, 2008 at 10:16 AM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 15, 2008, at 3:38 PM, Jean Krivine wrote: I'd be glad to try the patch if you could post it somewhere! I have posted it in: http://research.amnh.org/~avaron/ocaml/ best, Andres J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
I agree. I should use Int64 instead of just int, but I still think that the application (Random.int max_int) should not be exception prone. Since max_int is architecture dependent, then so should be Random.int no? But you point is well taken. Thanks again J On Wed, Jul 16, 2008 at 2:44 PM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 16, 2008, at 2:07 PM, Jean Krivine wrote: Good news, I just tested the patch and it works great with my application! I just had to modify the module random since a call to (Random.int max_int) may raise and exception (it is made for 32 bits integers). So I guess that modification should be included in the patch. I don't think that's a good idea. You have to use Random.int64 to get a 64 bit random integer. The Random.int function will return an integer between 0 and 2^30. Check the Random module documentation here: http://caml.inria.fr/pub/docs/manual-ocaml/libref/Random.html I wouldn't play with a random number generator unless I know exactly what I'm doing. Your results depend on it! (well, your messed-up-by-andres compiler could already have issues ... :-(, for what I use it I can verify the result with a 32 bit binary or a 64 bit linux binary, if you can, then do the same!). Andres Thanks a lot Andres. Jean On Wed, Jul 16, 2008 at 12:27 PM, Jean Krivine [EMAIL PROTECTED] wrote: Great thanks! J On Wed, Jul 16, 2008 at 10:16 AM, Andres Varon [EMAIL PROTECTED] wrote: On Jul 15, 2008, at 3:38 PM, Jean Krivine wrote: I'd be glad to try the patch if you could post it somewhere! I have posted it in: http://research.amnh.org/~avaron/ocaml/ best, Andres J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Re: [Caml-list] memory usage
Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] memory usage
I'd be glad to try the patch if you could post it somewhere! J On Tue, Jul 15, 2008 at 3:31 PM, Andres Varon [EMAIL PROTECTED] wrote: Hello Jean, There is no 64-bit native OCaml compiler for Mac OS X intel. I have a patch that works in Leopard, but did not compile opt.opt in Tiger, meaning that something is not OK, so I did not offer it to the community. The bootstrap went fine, findlib and godi compiled OK too. I can post the patches somewhere if you want to give it a shot. My memory intensive application runs fine in Leopard with this compiler. But the binaries do not execute in Tiger (I found that other people had the same trouble copying a 64 bit apps from Leopard to Tiger and the other way around, but didn't look into it). If you want it ... I can post it, maybe someone can cleanup my job? All that would be needed after patching is: ./configure -host x86_64-apple-darwin -prefix /opt/ocaml/experimental (The prefix I always add for my ocaml-modified comilers). best, Andres On Jul 15, 2008, at 1:06 PM, Jean Krivine wrote: Dear all I downloaded the last version of ocaml (3.10.2) but I must confess I don't know what option I should pass to the compiler to make a binary that uses 64 bits. I tried naively ocamlopt -ccopt -arch -ccopt x86_64 but that doesn't work. Any idea? On Fri, Jul 11, 2008 at 6:01 PM, Richard Jones [EMAIL PROTECTED] wrote: On Fri, Jul 11, 2008 at 03:49:26PM -0400, Jean Krivine wrote: I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: I can confirm that OCaml works fine with huge datasets, on 64 bit platforms anyway. sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Is the compiler 32 bits or 64 bits on this machine? Try doing: $ ocaml # Sys.word_size ;; It should print out either '32' or '64'. Also run your program under whatever the OS X equivalent of 'strace' is (ktrace?) to find out exactly why the mmap call fails. OCaml = 3.10.2 on Linux suffers a nasty problem with its use of mmap and randomized address spaces (https://bugzilla.redhat.com/show_bug.cgi?id=445545#c9) but it doesn't seem like this is the same issue. Rich. -- Richard Jones Red Hat ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] memory usage
Dear list members, I am trying to run a stochastic simulator (written in ocaml) on a huge data set and I have the following error message: sim(9595) malloc: *** mmap(size=1048576) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Fatal error: out of memory. My system: Mac Pro running OS X 10.5.4 Processor: 2 x 2.8 GHz Quad-Core Intel Xeon Memory: 10 GB 800 MHz DDR2 FB-DIMM Does someone know what happened? Do you have any idea of any parameter I could tune in order to avoid that? Thank you very much! Jean ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
