Re: [Caml-list] How does chroot work ?
Am Samstag, den 18.12.2010, 18:09 +0100 schrieb Gregory Bellier: Hi ! For security reasons, I would like to chroot a child process but I can't do it unless this process is root. How does it work exactly ? If everybody could chroot it would be possible to change passwords and do other privileged operations in the new chroot (it depends on the OS how dangerous this really is, but POSIX assumes it is dangerous). Because of this it is restricted to root. Furthermore, chroot is not designed for enhancing the security. A root process can undo chroot (look it up in the web, it's tricky but possible). If a normal user could chroot, everybody could also break out. So, usually you would start a new process as root, establish the chroot there, and setuid to a non-privileged user for doing the real work. If you cannot start as root, you could alternatively also set the setuid bit of the executable. However, running a process with setuid root adds new security dangers, so it is questionable whether you can improve the overall security by such means. I'd advise not to use chroot unless you exactly understand what you are doing. Gerd Thanks in advance. Gregory. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)
Am Dienstag, den 30.11.2010, 13:55 +0100 schrieb oli...@first.in-berlin.de: On Tue, Nov 30, 2010 at 09:10:36AM +0100, Stephan Houben wrote: On 11/29/2010 04:33 PM, Oliver Bandel wrote: Zitat von Gerd Stolpmann i...@gerd-stolpmann.de: Am Montag, den 29.11.2010, 17:12 +0100 schrieb Oliver Bandel: Zitat von Gerd Stolpmann i...@gerd-stolpmann.de: You use shared mem(?), but you link only to *.ml files, and I see no *.c there. How can this be done? At least not via the libs that are shipped with OCaml?! Actually it can be done using the libs that ship with OCaml (Unix and Bigarray), although it is not 100% POSIX : let create_shared_genarray kind layout dims = let fd = Unix.openfile /dev/zero [Unix.O_RDWR] 0 in let ar = Bigarray.Genarray.map_file fd kind layout true dims in Unix.close fd; ar The resulting bigarray object is shared among subsequent forks. Hmhhh... we started talking about Threads and SharedMem. You mean even fork hmhhh Independent processes are right now the only way to use several cores. You can organize shared memory between processes, but it is tricky. That's what I try to ease with my Netmulticore library. This relies on the fact that mmap-ing /dev/zero is equivalent to an anonymous mmap. http://en.wikipedia.org/wiki//dev/zero Well, at least it works on Linux. In APUE it's mentioned that memory mapped regions are inherited by a child, when forking it. So it should work on all Unix-systems too. Yes, but is not defined by POSIX what mapping /dev/zero means. There is one problem with this... when you have forked, then you obviously have separated processes and also in each process your own ocaml-program with it's own GC running... ..with such a mem-mapping trick (never used Bigarray, so I'm astouned it uses mmap) Bigarrays can use any memory with fixed addresses. That's the essence here: Bigarrays are not moved around by the GC. you then have independent processes, working on shared mem without synchronisation. This is a good possibility to get corrupted data, and therefore unreliable behaviour. So, you have somehow to create a way of communicating of these processes. So you need inter-process synchronization primitives, like POSIX semaphores. This already is easily done in the Threads-module, because synchronisation mechanisms are bound there to the OCaml API and can be used easily. In the Unix module there is not much of ths IPC stuff... But in Ocamlnet's netsys module. (A thread-specific GC for thread-specific variables would help here, making global locks only necessary when accessing global used variables. But I don't know if such a way would be possible without changing the GC-stuff itself.) The global lock does not protect user variables, but the Ocaml runtime, e.g. the state of the memory manager/garbage collector. Also it eases code generation - the memory image needs not be in a consistent state all the time (i.e. all pointers meaningful), but only when the runtime gets a hand on it. Removing this lock has far-reaching consequences. The oc4mc (ocaml for multicore) project used a separate minor heap per thread, which actually eases the task a lot - memory is in most cases allocated in the minor heap anyway. Many variables keeping the state of the runtime are then thread-local. Gerd Ciao, Oliver ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)
Am Dienstag, den 30.11.2010, 15:04 +0100 schrieb Stephan Houben: On 11/30/2010 12:55 PM, oli...@first.in-berlin.de wrote: There is one problem with this... when you have forked, then you obviously have separated processes and also in each process your own ocaml-program with it's own GC running... ...neatly sidestepping the problem that the GC needs to lock out all threads... .with such a mem-mapping trick (never used Bigarray, so I'm astouned it uses mmap) you then have independent processes, working on shared mem without synchronisation. This is a good possibility to get corrupted data, and therefore unreliable behaviour. Well, not more possibility than inherently in any code that updates a shared data structure in parallel. It is certainly not the case that the independently executing GCs in both processes are causing data corruption, since the GC only operates on unshared memory. Note that the GC doesn't move the Bigarray data around. (I am not sure if this was in particular your worry or that it was just the lack of synchronisation mechanisms which you bring up next. Apologies if I am addressing some non-concern.) So, you have somehow to create a way of communicating of these processes. This already is easily done in the Threads-module, because synchronisation mechanisms are bound there to the OCaml API and can be used easily. In the Unix module there is not much of ths IPC stuff... In fact there is the Unix.pipe function which can be used for message passing communication between processes. A pipe can also be used as a semaphore: operation V corresponds to writing a byte to the pipe, operation P corresponds to reading a byte. It's a bit heavy since it always makes a kernel call even for the non-contended case, but otherwise it works perfectly. For many purposes (e.g. something embarrassingly parallel like computing the Mandelbrot set) you can just divide the work up-front and only rely on the implicit synchronization given by waitpid. If you allow me a final observation/rant: I personally feel that the use of fork() and pipes as a way to exploit multiple CPUs is underrated. When appropriate (lots of computation and not so much synchronisation/communication) it works really great and is very robust because all data is process-private by default, as opposed to threading, where everything is shared and you have to stand on your head to get a thread-local variable. Performance can also be better since you don't run into cache coherency issues. I am not sure why it is not used more; possibly because it is not supported on Windows. I don't think this is the reason. Many people can ignore Windows, actually. The problem is more that your whole program needs then to be restructured - multi-processing implies a process model (which is the master, which are the workers). With multi-threading you can start threads at all times without having to worry about that (i.e. supports programming without design if you want to take that as a negative point). This is what I want to fix with my Netmulticore library - it defines a framework allowing you to start new processes at any time without having to worry about the process hierarchy. Also, many practical problems are only O(n log n), at most. The cost for serialization of data through a pipe cannot be neglected here. This makes shared memory attractive, even if it is only available in a restricted form (like write once memory). Gerd Stephan ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)
Am Dienstag, den 30.11.2010, 16:30 +0100 schrieb Stephan Houben: On 11/30/2010 02:22 PM, Gerd Stolpmann wrote: I don't think this is the reason. Many people can ignore Windows, actually. The problem is more that your whole program needs then to be restructured - multi-processing implies a process model (which is the master, which are the workers). With multi-threading you can start threads at all times without having to worry about that (i.e. supports programming without design if you want to take that as a negative point). This is what I want to fix with my Netmulticore library - it defines a framework allowing you to start new processes at any time without having to worry about the process hierarchy. I have in fact read with much interest your blog at http://blog.camlcity.org/blog/parallelmm.html . Your approach there is to really have separate programs for server and client. However, one nice thing about fork is that you don't have to restructure your program; you can just call fork down somewhere in some subroutine where you decide it is convenient, start doing some multicore computation, finish and return, and the caller needs never know that you did that. So you can indeed program without design using fork. Well, I would not recommend that in all cases: fork duplicates all memory, and if this is a lot, you can end up consuming a lot of RAM. Even worse, the GC of the forked subprocess has to manage all of the RAM, including the part that is not required for doing the computation in the subprocess (the copy-on-write optimization of the OS gets you nothing here). Also, there can be subtle interactions between the parent and the child, e.g. file descriptors are inherited, affecting whether closed descriptors can be recognized. So, use with care, and not without design. Forking in the middle of a bigger program can be quite disastrous. Of course, the advantage of your approach is that you can now distribute the work over multiple machines. So I guess there is an appropriate place for all of these techniques. I was recently working on an improved fork machinery, with some indirection between the request for creating a new worker process, and the actual fork. That's this netmulticore library I'm talking about. The new process is not a child of the process requesting the creation, but always a child of a common master process. This avoids all the problems (memory issues, file descriptor issues, and a few more), at the cost of having to transmit state to the new process. Also, many practical problems are only O(n log n), at most. The cost for serialization of data through a pipe cannot be neglected here. This makes shared memory attractive, even if it is only available in a restricted form (like write once memory). Well, the original context was one of a benchmark which had an arbitrary rule that you can only use functions from the bundled libraries. And my proposal was to use the pipe for synchronisation and the shared memory for bulk communication. If we drop the arbitrary rule there are faster options than pipes. (e.g. POSIX semaphores in a shared memory segment). Yes, it's really arbitrary. All remaining solutions are very restricted then, and don't have much to do with what you would choose for solving a real-world problem. This makes this benchmark irrelevant. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Is OCaml fast?
Am Sonntag, den 28.11.2010, 19:14 +0100 schrieb oli...@first.in-berlin.de: On Thu, Nov 25, 2010 at 11:50:58PM +0100, Fabrice Le Fessant wrote: [...] The main problem was that other languages have bigger standard libraries, whereas OCaml has a very small one (just what is needed to compile the compiler, actually). In many problems, you could benefit from using a very simple shared-memory library (in mandelbrot, the ocaml multicore solution has to copy the image in a socket between processes, whereas it could just be in a shared memory segment), ...so you work on a shared-mem module?! Don't know what Fabrice is referring to, but at least I work on a multicore-enabling library: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netmulticore/ This is work in progress and highly experimental. What's currently available: - managing processes and resources like files, shared memory objects etc. - support for message passing via Netcamlbox (another library) - low-level only so far: shared memory, including copying Ocaml values to and from shm - low-level only so far: kernel semaphores - a Netmulticore process is also a Netplex container, so mixing with Netplex-managed servers is possible. Also, the Netplex plugins are available (semaphores, mutexes, global variables), but these are relatively slow I've also written a few examples: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/examples/multicore/ Don't expect wonders, though. For instance, the port of the chameneos game is based on message passing, but this is by design slower than direct use of shared memory (and this game mostly consists of synchronization, and there is not any computation involved where multicore would be an advantage). The further direction is the addition of more primitives, especially for managing shared memory. The difficult here is that there is not any support in the core language, and I have to work around that. This is based on val Netsys_mem.init_value ?targetaddr:nativeint - ?target_custom_ops:(string * custom_ops) list - memory - int - 'a - init_value_flag list - (int * int) where type memory = (char,Bigarray.int8_unsigned_elt,Bigarray.c_layout) Bigarray.Array1.t This means shared memory is handled just as a bigarray of chars that is mapped into the address spaces of several processes. The init_value function copies an arbitrary Ocaml value to such a bigarray - in the same way as Marshal.to_string - only that it is to_bigarray, and that the same representation is used as for normal values (so you can access the copied values directly). This copying is quite time-consuming: you have to create the Ocaml value in normal memory first, and then to use a quite expensive generic copying machinery to get it to shared memory. It would be more elegant if there was a way to instruct ocamlopt so that code is emitted that creates the value directly in a user-supplied memory area. The arguments targetaddr and target_custom_ops point to another difficulty: For certain uses of shared memory you cannot ensure that all processes map the area to the same address. Because of this, there is support for filling bigarrays so that the addresses are offset for a different final mapping address. Netcamlbox uses this feature - the sending process can map the shared bigarray to any address, and nevertheless format a message with addresses that are right for the receiving process. In general, shared memory is difficult to add in an add-on library. However, some lessons can be learned, and maybe this results in some plot for adding better support in the core language. Gerd and in general, many solutions could benefit from specialised data structures that are provided in other languages by their standard libraries, and from some system calls that are currently not in the Unix library. [...] During the last some releases a lot more unix syscalls were added and that's fine of course). Which calls are you missing there? Ciao, Oliver ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: Threading and SharedMem (Re: [Caml-list] Re: Is OCaml fast?)
Am Montag, den 29.11.2010, 17:12 +0100 schrieb Oliver Bandel: Zitat von Gerd Stolpmann i...@gerd-stolpmann.de: Am Sonntag, den 28.11.2010, 19:14 +0100 schrieb oli...@first.in-berlin.de: On Thu, Nov 25, 2010 at 11:50:58PM +0100, Fabrice Le Fessant wrote: [...] The main problem was that other languages have bigger standard libraries, whereas OCaml has a very small one (just what is needed to compile the compiler, actually). In many problems, you could benefit from using a very simple shared-memory library (in mandelbrot, the ocaml multicore solution has to copy the image in a socket between processes, whereas it could just be in a shared memory segment), ...so you work on a shared-mem module?! Don't know what Fabrice is referring to, but at least I work on a multicore-enabling library: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netmulticore/ This is work in progress and highly experimental. What's currently available: - managing processes and resources like files, shared memory objects etc. - support for message passing via Netcamlbox (another library) - low-level only so far: shared memory, including copying Ocaml values to and from shm [...] You use shared mem(?), but you link only to *.ml files, and I see no *.c there. cd ../netsys it's part of a larger package How can this be done? At least not via the libs that are shipped with OCaml?! I would have expected some *.c for the shared mem part and the creation of Caml-values Ciao, Oliver P.S.: OCaml also provides a Thread-Lib, which seems to use pthread-lib. Normally this should help in making things possible to run on multiple cores. What are the restrictions that this does not run that way? Somehow... when all values are handled via one GC, then those threads are somehow bound together, but on the other side, it works threaded, and consumer-worker pipes and such stuff can be used. So... somehow the GC seems to be the point, where the show will be stopped? (Anyone who has looked inside OCaml here more detailed?) Quite easy: there is a global lock, and when Ocaml code runs, this lock must be acquired. So only one of the pthreads can have this lock, and so only one pthread can run Ocaml code. The reason is that memory management is not thread-safe. Gerd ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] [ANN] HTCaML / CaSS
Am Dienstag, den 23.11.2010, 15:01 +0100 schrieb Thomas Gazagnaire:
Have you seen that there is a preprocessor for PXP that allows you to
embed XML in ocaml?
http://projects.camlcity.org/projects/dl/pxp-1.2.1/doc/manual/html/ref/Intro_preprocessor.html
No, I didn't know about that. And there is also a syntax extension in
Eliom to do the same kind of things.
However, the two things I am really happy with HTCaML (and apparently it
is not possible to do the same thing in PXP nor Eliom) are :
i) you can easily mix auto-generated and hand-crafted code to create
XHTML fragments (no more tedious conversion functions); and
Don't know whether this is good or bad - having everything as XML means
you can also type-check the emitted XML (e.g. against the DTD). It's
only at runtime, but might still be helpful.
In PXP you would have to use a custom XML writer to merge in
hand-crafted HTML.
Btw, there are some subtle differences between real XHTML (with an
XML-ish content type) and HTML that looks like XML. For example, in
real XML SCRIPT or STYLE sections need to be quoted like other data
sections whereas HTML sees these as exceptions to the normal rules (e.g.
scriptif (a lt; b) { ... }/script in XML and scriptif (ab)
{...}/script in HTML). This is something that makes mixing in HTML
error-prone.
ii) you can write in the same part of your file (ie. in the same module)
the css and xhtml generator for a given type definition. That means that
you can do web-programming as you are used to : think about type
definitions first, and then write your code to reason by induction on
these defintions.
for example, you can have:
type foo = (* some random type *)
let html_of_foo : Html.t = (* some random code of type Html.t = (`a
Xml.frag as `a) Xmlm.frag list *)
let foo_css = (* some random, possibly nested, CSS *)
and :
type bar = { foo : foo; complex types } with html
let bar_css = :css .foo { $foo_css$; ... }
In the later case, you don't have to write manually the code for
html_of_bar as it will be done automatically by HTCaML, by looking at
the structure of bar; and it will pick your own definition of
html_of_foo. Also, nested declarations in bar_css will be automatically
unrolled to generate valid CSS fragments.
Interesting idea - data-centric HTML generation.
Gerd
I'm happily using this for dynamic web pages. The syntax is more
light-weight, though, e.g. you write
I don't really call this dynamic web pages, but yea, that's the same
idea :-)
--
Thomas
div [ ... ]
instead of div.../div, and there is a distinction in the syntax
between node and list of nodes, e.g.
div list
but
div [ node1 node2 ... ]
Gerd
Am Dienstag, den 23.11.2010, 14:05 +0100 schrieb Thomas Gazagnaire:
I am happy to announce the first official release of HTCaML[1] and
CaSS[2], two small libraries which make the writing of static web pages
easy in OCaml.
HTCaML enables the embedding of XHTML fragments in your OCaml program
(the EDSL translates directly to Xmlm) using quotations. It also allows
you to auto-generate boilerplate XHTML fragments from type definitions.
In the same way, CaSS enables the embedding of CSS fragments in your
OCaml program using quotations.
A quick example:
module Box = struct
type t = { title: string; date: string; contents: Html.t } with html
let css fg bg = :css
color: $fg$;
background-color: $bg$;
$Css.rounded$;
.title { color: $bg$; background-color: $fg$; }
end
let my_html boxes = Html.to_string :html
html
body
div class=boxes
$list:List.map Box.html_of_t boxes$
/div
/body
/html
let my_css = Css.to_string :css
.boxes { $Box.css :css blue :css white $ }
You can find a quick introduction to HTCaML (and maybe soon to CaSS) on
the mirage blog[3].
Cheers,
Thomas
[1] https://github.com/samoht/htcaml
[2] https://github.com/samoht/cass
[3] http://www.openmirage.org/blog/introduction-to-htcaml
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151
Re: [Caml-list] Re: Is OCaml fast?
Am Dienstag, den 23.11.2010, 17:53 + schrieb Isaac Gouy: Christophe TROESTLER Christophe.Troestler+ocaml at umh.ac.be writes: On Tue, 23 Nov 2010 02:03:48 +, Isaac Gouy wrote: C version : 12.11 secs OCaml version : 47.22 secs OCaml version with GC parameters tuned (interesting alternative section) : 12.67 secs And of course you know because that GC tuned OCaml program is shown on the benchmarks game website http://shootout.alioth.debian.org/u32/program.php?test=binarytrees; lang=ocamlid=1 Since you are here, please explain why C can use memory pools and vec tor instructions but tuning the GC of OCaml ― although it is part of the standard library ― is considered an “alternative”. You seem to be suggesting that tuning the GC is considered alternative only for OCaml programs. You seem to be suggesting that tuning the GC is considered alternative for every task. Neither is true. You seem to be suggesting some kind of equivalence between vector instructions and tuning the GC. You haven't said why they should be considered equivalent. Nor have you said why you think C should not be allowed to use memory pools. Quite easy: because you are comparing results that cannot be compared. The reader of this benchmark (binary trees) might have the impression that C is generally that fast - however, this would be no longer true if these binary trees were used as library in a bigger program where using memory pools is inappropriate, e.g. because the data managed by the binary trees has an unpredictable lifetime. I do not say that it is complete nonsense to do this comparison, but only that it is more specific than a reader would assume. The innocent reader expects a comparison of binary tree performance, not of methods of managing memory (and this is it finally). The true name of this test should be manage_many_small_memory_cells_where_pools_are_allowed. (It would be actually interesting to compare various versions of this test with different memory management methods.) Gerd ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ANN: ocamlnet-3.1
Hi list, A new version of ocamlnet (3.1) is ready for download and installation. This is mostly a bug fix release (see ChangeLog), but there is also a little extension in Netsys_mem: a function that creates a deep copy of an arbitrary value (copy_value). Download: http://download.camlcity.org/download/ocamlnet-3.1.tar.gz Manual: http://projects.camlcity.org/projects/dl/ocamlnet-3.1/doc/html-main/index.html Please report problems to g...@gerd-stolpmann.de GODI users: ocamlnet-3.1 is only provided for Ocaml 3.12, and in the ocamlnet3 overlay for Ocaml 3.11. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Is OCaml fast?
Am Dienstag, den 23.11.2010, 20:28 + schrieb Isaac Gouy: (It would be actually interesting to compare various versions of this test with different memory management methods.) So do that comparison and publish the results. Please don't tell me what I am supposed to do. I'm not a troll like others. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Is OCaml fast?
Am Montag, den 22.11.2010, 15:21 +0200 schrieb Thanassis Tsiodras: I apologize beforehand if this is not the forum to ask. I am on the fence about whether to learn OCaml or not, and while reading an article called Why OCaml (http://www.cs.ubc.ca/~murphyk/Software/Ocaml/why_ocaml.html), I saw that OCaml was praised for the speed of the executables it generates - and was referred to, speed-wise, as second to none, except C and C++. However, when I actually went to the Language Shootout page suggested in the article, I found out that OCaml is not 2nd, it is 13th, behind languages like Haskell and C#... (http://shootout.alioth.debian.org/u32/which-programming-languages-are-fastest.php) Is it just hype, then? Or am I missing something? I think the shootout is not a good data source. There are definitely some very poor Ocaml results there, so I'd guess the shootout got recently more attention by enthusiasts of other languages, and the current Ocaml programs there are not very good. (I remember Ocaml was #1 at the shootout a few years ago, faster than C.) So maybe a good opportunity to post better Ocaml solutions there? Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Is OCaml fast?
Am Montag, den 22.11.2010, 15:36 +0100 schrieb bluestorm: On Mon, Nov 22, 2010 at 3:04 PM, Gerd Stolpmann i...@gerd-stolpmann.de wrote: I think the shootout is not a good data source. There are definitely some very poor Ocaml results there, so I'd guess the shootout got recently more attention by enthusiasts of other languages, and the current Ocaml programs there are not very good. (I remember Ocaml was #1 at the shootout a few years ago, faster than C.) So maybe a good opportunity to post better Ocaml solutions there? As Sylvain noticed, some (in not most) of the OCaml poor performances in the shootout are actually not due to bad OCaml programs, but to arbitrary restrictions in the shootout rules. For example, one of the bad-performing benchmark for OCaml is the binary-tree benchmark, where it is nearly four times slower than C, but on closer inspection you discover that this is due to the arbitrary choice to forbid any change of the GC parameters. With appropriate GC parameters, the very same OCaml program is exactly as fast as C. http://shootout.alioth.debian.org/u32/performance.php?test=binarytrees « Note: these programs are being measured with the default initial heap size - the measurements may be very different with a larger initial heap size or GC tuning. » C version : 12.11 secs OCaml version : 47.22 secs OCaml version with GC parameters tuned (interesting alternative section) : 12.67 secs Therefore, there is nothing that can be changed to the OCaml submission for this benchmark to improve performances, except changing the default GC parameters; while this might be a good idea in general, changing it only for the sake of shootout-obsessed people is ridiculous. It's in deed an unfair comparison: In C they use the Apache runtime which provides memory pools. This is something that does not extend to most real world programs. Because it's ridiculous anyway: Encode the tree in an array. Not really idiomatic, but in C they also do not use the idiomatic memory management (malloc/free). Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] [Was: OCamlJit 2.0]
Am Samstag, den 20.11.2010, 08:10 -0800 schrieb Yoann Padioleau: On Nov 20, 2010, at 7:42 AM, Benedikt Meurer wrote: On Nov 20, 2010, at 16:19 , Vincent Balat wrote: On Nov 19, 2010 21:20:01, Yoann Padioleau wrote: On Nov 19, 2010, at 11:46 AM, Dario Teixeira wrote: ... Actually, Facebook has a compiler that transforms PHP source code into C++ [1], and they claim a 50% reduction in CPU usage. Yes, which is good. But if you think about it is only a x2 speedup vs a really slow bytecode interpreter (the Zend PHP interpreter). PHP is known for being more than 30 times slower than C. It's even slower than Ruby on http://shootout.alioth.debian.org/u32/which-programming-languages-are-fast est.php There are lots of opportunities to do better IMHO. Would it be completely inconceivable for a company like facebook to reimplement everything using a fast well designed typed language I think it is inconceivable. I doubt facebook will switch to ocaml and ocsigen tomorrow :) The problem is how to migrate code to another language smoothly ? You can not stop all development for a month and say Hey everybody, we are porting our 10 millions lines of code of PHP to X.. The reason C++ succeeded was because there was a smooth migration path. C code is valid C++ code. You can incrementally add objects to an existing codebase. It's not a matter of the technical migration. You simply wouldn't find enough engineers that can master a fast well designed typed language (assuming here you have the more advanced ones in mind :-) ). Also, there is the question whether it is the best approach to let the most talented developers in the company program the user interface, with day-to-day tasks like hey Joe, can you please fix this broken link. (Not that there are also complicated UI's but these are rare.) At Mylife.com (where I'm currently consulting for) large parts of the backend are now running in Ocaml, but the web frontend is done by another team, and there is PHP involved. IMHO, it would be a misallocation of human resources if we (the Ocaml team) also did the frontend. Do we have example of big companies porting their whole codebase to another language ? Usually languages are only switched when a completely new project is started. Gerd instead of spending hundreds of millions of dollars on machines that run PHP bytecode interpreters? (quoting Yoann) Apparently they decided to keep PHP and switch from a slow bytecode interpreter to a 2x-faster compiler. It's probably not a technical decision, but more likely a marketing decision. If you tell Joe Who is Joe ? A developer ? A user ? A venture capitalist ? that your webservices run on Java, PHP or .NET, he'll say great, sure or wow (not because Joe's familiar with the technology or the theory, but because he's familiar with the terms). Tell Joe your webservices run on OCaml or Haskell and the best answer you can get will be what?. I doubt any user care about how facebook is implemented. Twitter and Foursquare run on Scala and this is not a very popular language. Vincent [not completely joking] Benedikt ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] SMP multithreading
) and value sharing. (That's probably the path I'll follow for my own experiments - no modifications of the runtime, but play tricks with the OS.) * As somebody mentioned implicit parallelization: Don't expect anything from this. Even if a good compiler finds ways to parallelize 20% of the code (which would be a lot), the runtime effect would be marginal. 80% of the code is run at normal speed (hopefully) and dominates the runtime behavior. The point is that such compiler-driven code improvements are only local optimizations. For getting good parallelization results you need to restructure the design of the program - well, maybe compiler2.0 can do this at some time, but this is not in sight. * Looking for more automatic speedups: I would more look for parallelizing parts of the GC (e.g. parallelized sweep), but this is probably running quickly against the memory bandwidth limit. Maybe using 2 cores for the GC would result in an improvement, and more cores get you nothing extra. At least worth an experiment. Gerd E. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] SMP multithreading
Am Dienstag, den 16.11.2010, 22:35 +0200 schrieb Eray Ozkural: On Tue, Nov 16, 2010 at 7:04 PM, Gerd Stolpmann i...@gerd-stolpmann.de wrote: Am Montag, den 15.11.2010, 22:46 -0800 schrieb Edgar Friendly: * As somebody mentioned implicit parallelization: Don't expect anything from this. Even if a good compiler finds ways to parallelize 20% of the code (which would be a lot), the runtime effect would be marginal. 80% of the code is run at normal speed (hopefully) and dominates the runtime behavior. The point is that such compiler-driven code improvements are only local optimizations. For getting good parallelization results you need to restructure the design of the program - well, maybe compiler2.0 can do this at some time, but this is not in sight. I think you are underestimating parallelizing compilers. I was more citing Amdahl's law, and did not want to criticize any effort in this area. It's more the usefulness for the majority of problems. How useful is the best parallelizing compiler if only a small part of the program _can_ actually benefit from it? Think about it. If you are not working in an area where many subroutines can be sped up, you consider this way of parallelizing as a waste of time. And this is still true for the majority of problems. Also, for many problems that can be tackled, the scalability is very limited. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] rpc: type conversion
Alternatively, you can instruct ocamlrpcgen to directly translate RPC
int's to int/int32/int64. E.g. ocalmrpcgen -int unboxed, or
you demand the types in the .x file, e.g. use _unboxed int.
See the manual for details:
http://projects.camlcity.org/projects/dl/ocamlnet-3.0.3/doc/html-main/Rpc_intro.html,
section Mapping integer types.
Gerd
Am Freitag, den 05.11.2010, 00:15 +0100 schrieb Philippe Strauss:
oops, Rtypes.int_of_int4 and int4_of_int.
sorry.
Philippe Strauss wrote:
Hello,
I need to use RPC in a current project of mine, but I don't get how do
you write the type conversion and what is provided, taking a very
basic example:
--8--
program Rpc_control {
version V1 {
int add(int, int)= 1;
} = 1;
} = 0x207f;
--8--
and the server code:
--8--
let add arg =
match arg with
| (a, b) - a + b
| _ - 0
let _ =
let esys = Unixqueue.standard_event_system () in
let server = Deconv_srv.Rpc_control.V1.create_server ~proc_add: add
(Rpc_server.Localhost ) Rpc.Tcp Rpc.Socket esys in
Unixqueue.run esys
--8--
my add fun is wrong regarding the types, I've read about type
conversion but the doco lacks examples, the examples lacks basic use
of int - Rtype.int4.
Thanks
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] [ANN] Plasma MapReduce, PlasmaFS, version 0.2
Hi, I've just released Plasma-0.2. Plasma consists of two parts (for now), namely Plasma MapReduce, a map/reduce compute framework, and PlasmaFS, the underlying distributed filesystem. Major changes in version 0.2 : * Enhanced filesystem operations, with enhanced locking * Full support for symbolic links * Extended filesystem API, both in the ocaml module and the command-line utility * One can now mount filesystems read-write via NFS * Map/reduce processes files in larger chunks than before * Streaming (as in Hadoop) * Support for per-task logging Of course, there are also numerous bug fixes and performance improvements. Plasma MapReduce is a distributed implementation of the map/reduce algorithm scheme. In a sentence, map/reduce performs a parallel List.map on an input file, sorts and splits the output by some criterion into partitions, and runs a List.fold_left on each partition. Only that it does not do that sequentially, but in a distributed way, and chunk by chunk. Because of this Plasma MapReduce can process very large files, and if run on enough computers, this also will work in reasonable time. Of course, map and reduce are Ocaml functions here. This all works on top of a distributed filesystem, PlasmaFS. This is a user-space filesystem that is primarily accessed over RPC (but it is also mountable as NFS volume). Actually, most of the effort went here. PlasmaFS focuses on reliability and speed for big blocksizes. To get this, it implements ACID transactions, replicates data and metadata with two-phase commit, uses a shared memory data channel if possible, and monitors itself. Unlike other filesystems for map/reduce, PlasmaFS implements the complete set of usual file operations, including random reads and writes. It can also be used as unspecialized global filesystem. Both pieces of software are bundled together in one download. The project page with further links is http://projects.camlcity.org/projects/plasma.html This is an early alpha release (0.2). A lot of things work already, and you can already run distributed map/reduce jobs. However, it is in no way complete. Plasma is installable via GODI for Ocaml 3.12. For discussions on specifics of Plasma there is a separate mailing list: https://godirepo.camlcity.org/mailman/listinfo/plasma-list Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] web server interface for Ocaml ( like rack, wsgi, ...)?
Am Dienstag, den 21.09.2010, 20:20 +0100 schrieb Richard Jones:
On Tue, Sep 21, 2010 at 09:50:36AM +0200, ben kuin wrote:
what's wrong with simply proxying the HTTP connections through your
favorite webserver to the backend ocsigen/ocamlnet server?
Nothing per se. I've tried so set something up with ocsigen, but I had
to give up. There seems to be a certain philosophy behind it how you
should writing web apps.
But I just wanted to do something:
- small ( one page app )
- quick and dirty ( no big setup upfront, no compilation)
- deployment, by dropping a (ocaml script) into a webroot, not matter
what webserver is running
- permissive license ( mit, ...)
from wikipedia
require 'rack'
app = Rack::Builder.app do
lambda do |env|
body = Hello, World!
[200, {Content-Type = text/plain, Content-Length =
body.length.to_s}, [body]]
end
end
This ought to work, and it's pretty simple ...
-
#!/usr/bin/ocamlrun ocaml
print_string \
Content-Type: text/plain
Hello, World!
-
'course the problem is you'll be wanting to parse parameters, cookies
and the rest of it, and there it does get a little bit more
complicated.
I'm fairly sure ocamlnet can write standalone scripts like that? Gerd??
#! /opt/godi-3.11/bin/ocamlrun /opt/godi-3.11/bin/ocaml
#use topfind;;
#require netcgi2;;
open Netcgi
let () =
Netcgi_cgi.run
~output_type:(`Direct )
(fun cgi -
let body = Hello, World! in
cgi # set_header
~content_type:text/plain
~content_length:(String.length body)
();
cgi # out_channel # output_string body;
cgi # out_channel # commit_work()
)
Gerd
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: ancient module
Am Dienstag, den 14.09.2010, 21:46 +0100 schrieb Richard Jones: On Tue, Sep 14, 2010 at 08:19:49PM +, Yoann Padioleau wrote: Hi, I am trying to use your Ancient module to avoid having the garbage collector spends lots of time iterating over huge data in memory. It works quite well for arrays but for hashtbl I have some problems where I am not able to find back keys that were clearly in the original hashtbl (before Ancient.mark it). In the doc it says: (1) Ad-hoc polymorphic primitives (structural equality, marshalling and hashing) do not work on ancient data structures, meaning that you will need to provide your own comparison and hashing functions. The issue is described by Xavier Leroy: http://caml.inria.fr/pub/ml-archives/caml-list/2006/09/977818689f4ceb2178c592453df7a343.en.html As far as my understanding goes, what happens is that the OCaml compare function (or some C equivalent in the runtime) looks at the two string pointers and decides that since both are out of the normal heap they are just opaque objects. Thus it won't compare the content of the strings, but will just do pointer equality. This massively breaks assumptions in some ordinary OCaml code, in this instance in Hashtbl. There is now a way to change this. You can call caml_page_table_add (since 3.11) to explicitly declare a memory region as containing Ocaml values. The polymorphic comparison, the hash primitive, and marshalling work then. There is support for this in Ocamlnet-3: http://projects.camlcity.org/projects/dl/ocamlnet-3.0.3/doc/html-main/Netsys_mem.html#VALvalue_area Gerd which mean I have to transform my code using Hashtbl.xxx into one using the functorized version of hashtbl ? I have hashtbl of strings to complex data type. What would be a good hash function for strings ? It may be that Map also has the same problems. You wouldn't really know except by examining the code. Later you wrote: Actually it seems I have the problem only with Hashtbl from strings to whatever. I also have some Hashtbl from int to whatever and they work fine after the Ancient.mark. ints aren't compared in the same way. They are always compared using pointer equality, so there's no issue. I've only used ancient to store simple arrays, and when we needed to do string equality I remember writing a function which was aware of the above issue (you can compare them byte for byte just fine, even from OCaml code). Rich. -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] ANN: ocamlnet-3.0.0
Hi, unfortunately there were two bugs in this version (one build bug, one serious race condition), both fixed in 3.0.1 (available from the same place). Also, I wrote an/another article about the changes in Ocamlnet 3: http://blog.camlcity.org/blog/ocamlnet3_release.html Hope you like it. Gerd Am Dienstag, den 31.08.2010, 18:58 +0200 schrieb Gerd Stolpmann: Hi list, I'm very proud to announce Ocamlnet 3.0.0, a completely overhauled version of Ocamlnet. List of major changes: * Port to Win32 (as outlined in the blog article http://blog.camlcity.org/blog/ocamlnet3_win32.html) * The new Rpc_proxy layer (as described in http://blog.camlcity.org/blog/ocamlnet3_ha.html) * Extensions of Netplex (Netplex_sharedvar etc.) * New implementation of the Shell library for starting subprocesses * Uniform debugging with Netlog.Debug * Exception printers (Netexn) * Coordination of signal handling in Netsys_signal * New foundation for Unixqueue via pollsets * Extended Unixqueue engines (e.g. Uq_io) * More system calls in netsys * Camlboxes as an efficient way of message passing between processes * The netcgi1 library has been dropped in favor of netcgi2 There are also a lot of minor changes. Some of the changes are incompatible with code written for Ocamlnet 2, but experience shows that this rarely creates problems. Download: http://download.camlcity.org/download/ocamlnet-3.0.0.tar.gz Manual: http://projects.camlcity.org/projects/dl/ocamlnet-3.0.0/doc/html-main/index.html Please report problems to g...@gerd-stolpmann.de GODI users: For the still inofficial 3.12 release of GODI ocamlnet-3.0.0 will be the regular version of Ocamlnet. For the 3.11 release it is planned to stick to ocamlnet-2.2.9, and to offer an optional upgrade to ocamlnet-3.0.0. (See separate message in godi-list.) Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ANN: ocamlnet-3.0.0
Hi list, I'm very proud to announce Ocamlnet 3.0.0, a completely overhauled version of Ocamlnet. List of major changes: * Port to Win32 (as outlined in the blog article http://blog.camlcity.org/blog/ocamlnet3_win32.html) * The new Rpc_proxy layer (as described in http://blog.camlcity.org/blog/ocamlnet3_ha.html) * Extensions of Netplex (Netplex_sharedvar etc.) * New implementation of the Shell library for starting subprocesses * Uniform debugging with Netlog.Debug * Exception printers (Netexn) * Coordination of signal handling in Netsys_signal * New foundation for Unixqueue via pollsets * Extended Unixqueue engines (e.g. Uq_io) * More system calls in netsys * Camlboxes as an efficient way of message passing between processes * The netcgi1 library has been dropped in favor of netcgi2 There are also a lot of minor changes. Some of the changes are incompatible with code written for Ocamlnet 2, but experience shows that this rarely creates problems. Download: http://download.camlcity.org/download/ocamlnet-3.0.0.tar.gz Manual: http://projects.camlcity.org/projects/dl/ocamlnet-3.0.0/doc/html-main/index.html Please report problems to g...@gerd-stolpmann.de GODI users: For the still inofficial 3.12 release of GODI ocamlnet-3.0.0 will be the regular version of Ocamlnet. For the 3.11 release it is planned to stick to ocamlnet-2.2.9, and to offer an optional upgrade to ocamlnet-3.0.0. (See separate message in godi-list.) Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] OCaml 3.12.0+beta1 on Cygwin 1.7.5
Am Dienstag, den 06.07.2010, 17:36 -0400 schrieb Taylor Venable: Hi there, I'm thinking of writing some programs in OCaml to assist a custom build infrastructure (currently 95% Tcl, 5% external Lua programs) that we use at work. Since we build on Windows, Mac, and Linux I have to make sure that whatever I end up writing works in all three. I haven't written much OCaml in my life, and the little that I did was probably five years ago, but any tools I write will have to be usable for a while. So I tried compiling OCaml 3.12.0+beta1 in Cygwin 1.7.5 -- the information I found about it seemed to imply that it should work. I put flexdll in the PATH and I was able to use the configure script and make world.opt with only a single problem: there were lots of linker errors when it came to building tools/objinfo_helper.exe, symbols like _lbasename, _sch_istable, and _objalloc_free could not be found. [I'd paste the full error but I forgot to email it to my private address today and the machine is in corporate lockdown mode, so I can't access it right now, but I can obtain it tomorrow at work if anybody would like to see the full message.] The way to fix it, for me anyway, was to modify the definition of LIBBFD_LINK in config/Makefile (which started out as -lbfd -ldl) to append to it -lintl -lz /usr/lib/libiberty.a; and after that everything worked. I don't know if that constitutes a problem with the configure script or just something about how my Cygwin is installed, but I wanted to mention it somewhere in case it was a bug. In addition to that, I have a question: is bytecode compatible across operating systems and/or CPU architectures? In other words, if I compile to bytecode on Linux/x86, can I run that using ocamlrun on Windows/x86 or Linux/amd64? Trivial programs: yes. (As trivial as the ocaml compiler which uses the same bytecode to bootstrap on all platforms.) There is a problem with some libraries, however, because they differ by platform, for example the unix library (i.e. unix.cma is different on Linux and Windows). You can try to work around by linking ad-hoc, but you need the ocaml executable then, e.g. ocaml unix.cma your_library.cma your_script.ml which then links with the unix.cma that is locally installed, and immediately runs the program. At least the module checksums for unix.cma should be identical on all platforms. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Problems with Godi and Expat under Lucid Lynx
Am Donnerstag, den 01.07.2010, 11:34 +0200 schrieb Till Crueger: Hi, I am trying to rebuild ocaml-expat in Godi after upgrading to Lucid Lynx and have encountered some problems. The build process in the Godi console fails with the message: === expat not found Consider adding GODI_BASEPKG_EXPAT=yes to godi.conf Error: Exec error: File /opt/godi/build/conf/conf-expat/./../../mk/bsd.pkg.mk, line 1022: Command returned with non-zero exit code I hade some look at the problem and found the following. The configure script from godi is trying to compile a small c-program to test for the existing of libexpat. The program compiles fine and even runs, when I execute it directly. However the directories do not match the patterns desired by the configure script. All development files are laid out in the following way under Lucid Lynx: Headers of expat in: /usr/lib/expat.h Libraries: /lib/libexpat.so.1.5.2 /lib/libexpat.so.1 - /lib/libexpat.so.1.5.2 /usr/lib/libexpat.a /usr/lib/libexpat.so - /lib/libexpat.so.1.5.2 running ldd on the binary tells me, that the libexpat.so.1 from the directory /lib is used in this program. The script however expects the libdir to be /usr/lib (which would work with one of the links). Because of this mismatche it keeps on trying other directories and eventually fails. Well, maybe it should also look into /lib when it finds the .h in /usr/include. Any ideas on how to work around this in a simple way? Right now my solution will probably be to install an aditional copy of libexpat under /opt and use that one for ocaml. However I don't really want to keep duplicate packages around all the time, just to make it work. Just configure it manually: For the package conf-expat set the variables: GODI_EXPAT_INCDIR = /usr/include GODI_EXPAT_LIBDIR = /lib (type 'c' to set this after going to the conf-expat package). Gerd On the other hand I am not really happy with the way the expat binding works in Ocaml, so I might just switch to another XML parser for my code soon. Any good suggestions for a simple to use XML parser library for OCaml? Thanks, Till ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Problems with Godi and Expat under Lucid Lynx
Should also be fixed now in conf-expat. Gerd Am Donnerstag, den 01.07.2010, 11:34 +0200 schrieb Till Crueger: Hi, I am trying to rebuild ocaml-expat in Godi after upgrading to Lucid Lynx and have encountered some problems. The build process in the Godi console fails with the message: === expat not found Consider adding GODI_BASEPKG_EXPAT=yes to godi.conf Error: Exec error: File /opt/godi/build/conf/conf-expat/./../../mk/bsd.pkg.mk, line 1022: Command returned with non-zero exit code I hade some look at the problem and found the following. The configure script from godi is trying to compile a small c-program to test for the existing of libexpat. The program compiles fine and even runs, when I execute it directly. However the directories do not match the patterns desired by the configure script. All development files are laid out in the following way under Lucid Lynx: Headers of expat in: /usr/lib/expat.h Libraries: /lib/libexpat.so.1.5.2 /lib/libexpat.so.1 - /lib/libexpat.so.1.5.2 /usr/lib/libexpat.a /usr/lib/libexpat.so - /lib/libexpat.so.1.5.2 running ldd on the binary tells me, that the libexpat.so.1 from the directory /lib is used in this program. The script however expects the libdir to be /usr/lib (which would work with one of the links). Because of this mismatche it keeps on trying other directories and eventually fails. Any ideas on how to work around this in a simple way? Right now my solution will probably be to install an aditional copy of libexpat under /opt and use that one for ocaml. However I don't really want to keep duplicate packages around all the time, just to make it work. On the other hand I am not really happy with the way the expat binding works in Ocaml, so I might just switch to another XML parser for my code soon. Any good suggestions for a simple to use XML parser library for OCaml? Thanks, Till ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] [ANN] Plasma MapReduce, PlasmaFS
Hi, I'm very proud to announce the public availability of Plasma MapReduce, a map/reduce compute framework, and PlasmaFS, the underlying distributed filesystem. Plasma MapReduce is a distributed implementation of the map/reduce algorithm scheme. In a sentence, map/reduce performs a parallel List.map on an input file, sorts and splits the output by some criterion into partitions, and runs a List.fold_left on each partition. Only that it does not do that sequentially, but in a distributed way, and chunk by chunk. Because of this Plasma MapReduce can process very large files, and if run on enough computers, this also will work in reasonable time. Of course, map and reduce are Ocaml functions here. This all works on top of a distributed filesystem, PlasmaFS. This is a user-space filesystem that is primarily accessed over RPC (but it is also mountable as NFS volume). Actually, most of the effort went here. PlasmaFS focuses on reliability and speed for big blocksizes. To get this, it implements ACID transactions, replicates data and metadata with two-phase commit, uses a shared memory data channel if possible, and monitors itself. Unlike other filesystems for map/reduce, PlasmaFS implements the complete set of usual file operations, including random reads and writes. It can also be used as unspecialized global filesystem. Both pieces of software are bundled together in one download. The project page is http://projects.camlcity.org/projects/plasma.html This is an early alpha release (0.1). A lot of things work already, and you can already run map/reduce jobs. However, it is in no way complete. For discussions on specifics of Plasma there is a separate mailing list: https://godirepo.camlcity.org/mailman/listinfo/plasma-list Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Shared memory parallel application: kernel threads
/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ANN: ocamlnet-3.0test2
Hi list, it is my pleasure to announce the second test version of the upcoming ocamlnet3 library. This version mainly includes a large number of bug fixes compared to the first version, but also a few additions: * netcamlbox: a fast ipc mechanism for sending ocaml values to another process. Netcamlbox is shared-memory based, and works well on multi-cores (see http://projects.camlcity.org/projects/dl/ocamlnet-3.0test2/doc/html-main/Netcamlbox.html for doc) * netplex adds per-process sockets, so one can send messages to individual containers, and not only to the whole service * wrappers for POSIX semaphores * wrappers for syslog * performance optimizations (serialization, page-aligned I/O) * updated documentation Already in the first test version: * Port to Win32 (as outlined in the blog article http://blog.camlcity.org/blog/ocamlnet3_win32.html) * The new Rpc_proxy layer (as described in http://blog.camlcity.org/blog/ocamlnet3_ha.html) * Extensions of Netplex (see especially http://blog.camlcity.org/blog/ocamlnet3_mp.html) * New implementation of the Shell library for starting subprocesses * Uniform debugging with Netlog.Debug * Exception printers (Netexn) * Introduction of pollsets (Netsys_pollset); removal of Unix.select (i.e. more than 1024 file descriptors) * The netcgi1 library has been dropped in favor of netcgi2 I've quickly checked that the library builds on linux, freebsd-7.2, open solaris, and Win32 (MinGW). Nevertheless, testers are especially encouraged to check whether Ocamlnet 3 still works on all platforms, because a lot of new platform-specific code has been added. Download etc: * Homepage: http://projects.camlcity.org/projects/ocamlnet.html * Download: http://download.camlcity.org/download/ocamlnet-3.0test2.tar.gz * Manual: http://projects.camlcity.org/projects/dl/ocamlnet-3.0test2/doc/html-main/index.html * My scratch pad describing changes, plans, etc: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/TODO * Subversion: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/ There is a GODI package, but you have to enable a special repository to get it: Add GODI_BUILD_SITES+=http://www.ocaml-programming.de/godi-build/ocamlnet3/ to godi.conf to see the new packages in godi_console. This works first after the bootstrap is finished (godi_console cannot be built with ocamlnet3 yet). Keep in mind that this is development code, and there is no easy way to downgrade to ocamlnet2. Best is you do this only for new GODI installations. Special thanks to everybody who helped me to produce this new version - by reporting bugs, or even sending fixes, or by maintaining subtrees (Christophe Troestler). Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] The need to specify 'rec' in a recursive function defintion
Am Dienstag, den 09.02.2010, 15:50 -0500 schrieb Saptarshi Guha: Hello, I was wondering why recursive functions need to be specified with rec. According to Practical Ocaml, to inform the compiler that the function exists. But when entering the function definition, can't the compiler note that the function is being defined so that when it sees the function calling itself, it wont say Unbound value f? How is the knowledge of a function being rec taken advantage of (in ocaml) as opposed to other languages (leaving aside tail call optimization). Wouldn't one of way of detecting a recursive function would be to see if the indeed the function calls itself? Sure, but that's a purely syntactical point of view. In the ML community it is consensus that a recursive function is a total different thing than a non-recursive function. The rec is just the syntactical expression of this differentiation. Keep in mind that let f arg = expr is just a short-hand notation for let f = (fun arg - expr) or, in other words, the anonymous function constructor (fun arg - expr) is the basic building block to which the let construction is broken down. The anonymous function has a direct counterpart in the lambda calculus, i.e. this is the level of mathematical groundwork. You cannot directly express recursion in an anonymous function. For defining the operational meaning of a recursive function a special helper is needed, the Y-combinator. It gets quite complicated here from a theoretical point of view because the Y-combinator is not typable. But generally, the idea is to have a combinator y that can be applied to a function like y (fun f arg - expr) arg and that runs this function recursively, where f is the recursion. let rec is considered to be just a short-hand notation for using y. Besides the different way of defining let and let rec there are also differences in typing. Gerd These are very much beginners' questions. Thank you Saptarshi ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] The need to specify 'rec' in a recursive function defintion
Am Dienstag, den 09.02.2010, 22:58 +0100 schrieb Guillaume Yziquel: Gerd Stolpmann a écrit : Am Dienstag, den 09.02.2010, 15:50 -0500 schrieb Saptarshi Guha: Besides the different way of defining let and let rec there are also differences in typing. Well, at least you can have new effects. Look for polymorphic recursion. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Controlling module loading order.
Hi Guillaume, if you want to control from findlib that a certain function is invoked, the usual way to do it is to put a .cmo/.cmx file into the archive variables. The problem is that the linker drops all unused modules from .cma/.cmxa archives, and as a consequence the initialization code of these modules is not executed. So you could make R.interpreter a .cmo/.cmx - in this case the module is always initialized. The other workaround is to provide an init function in R.interpreter like let init() = () By calling this function the user references the interpreter, and all the initialization code is executed. Gerd Am Freitag, den 08.01.2010, 21:54 +0100 schrieb Guillaume Yziquel: Hi. I've been reimpleminting the OCaml-R binding, and implemented a simple wrapper around the Quantmod package in R: http://yziquel.homelinux.org/gitweb/?p=ocaml-r.git;a=tree http://yziquel.homelinux.org/gitweb/?p=ocamlr-quantmod.git;a=tree Testing these modules from the toplevel is quite fine. However, when compiling stuff using these pieces of code, I have issues with the way the modules are loaded, since the order in which they are loaded has side-effects: Initialisation of the R interpreter in the good case, segfaults in the bad case... For instance, the META file of OCaml-R: 1 name = R 2 version = 0.2 3 description = R bindings for OCaml 4 requires = unix 5 archive(byte) = r.cma 6 archive(native) = r.cmxa 7 8 package interpreter ( 9 version = 0.2 10 description = Embedded R interpreter 11 requires = R 12 archive(byte) = oCamlR.cma 13 archive(native) = oCamlR.cmxa 14 ) The stub functions are in package R, and package R.interpreter contains a module with and empty signature, whose side-effects are to initialise the R interpreter through an application of the functor 19 module Interpreter (Env : Environment) : Interpreter = struct 20 21 let () = init ~name: Env.name 22 ~argv: Env.options 23 ~env: Env.env 24 ~sigs: Env.signal_handlers 25 () 26 27 end My issue concerns the Quantmod wrapper: How can I make sure that when the Quantmod module is loaded, the OCamlR module of the R.interpreter findlib package gets loaded before? Currently the ocamlbuild _tags file for ocamlr-quantmod is 1 quantmod.ml: pkg_R.interpreter, pkg_CRAN-zoo But that doesn't seem to do the trick... My question is: do I have to put a line like module X = OCamlR in quantmod.ml, or is there a way to load OCamlR beforehand just by tweaking the build process, order of modules when linking, etc... All the best, -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] problem creating .cma library
For instance, here is the piece of code that executes R code ad catches
errors. I've tried to follow guidelines available on the net, and
inspired myself from pcre.
CAMLprim value r_eval_sxp (value sexp_list) {
CAMLparam0();
CAMLlocal2(ml_error_call, ml_error_message);
SEXP e;
int error = 0;
PROTECT(e = R_tryEval(Sexp_val(sexp_list), R_GlobalEnv, error));
UNPROTECT(1);
if (error) {
ml_error_call = Val_sexp(error_call);
error_call = NULL;
ml_error_message = caml_copy_string(error_message);
error_message = NULL;
value error_result = caml_alloc_small(2, 0);
Store_field(error_result, 0, ml_error_call);
Store_field(error_result, 1, ml_error_message);
raise_with_arg(*caml_named_value(OCaml-R generic error),
error_result);
}
CAMLreturn(Val_sexp(e));
}
Do you see GC / allocation / threading problems with it?
Would it be legitimate to include CAMLlocal2 inside the error-handling
braces?
No. You would start a new context for local roots, and there is no way
to end it (CAMLreturn ends the context).
There are the macros Begin_rootsn and End_roots that should be used in
this case, e.g.
if (error) {
value ml_error_call = Val_unit;
value ml_error_message = Val_unit;
Begin_roots2(ml_error_call, ml_error_message);
...
End_roots();
raise_with_arg(...)
}
The macros are only documented in memory.h.
So far I know, raising an exception from within Begin_roots/End_roots is
not allowed.
Gerd
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: multicore wish
Am Montag, den 28.12.2009, 19:05 +0100 schrieb Xavier Leroy: Gerd Stolpmann wrote: It works with all types: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netsys/netsys_mem.mli look for init_value. It's non-released code yet. However, there are some problems: Values outside the heap do not support the polymorphic comparison and hash functions. That's a hard limitation, e.g. you cannot even compare two strings, or build a hash table with strings as keys. That limits the usefulness of shared memory. In OCaml 3.11 and later, you can call caml_page_table_add(In_static_area, start, end) to inform the run-time system that the address range [start, end) contains well-formed Caml data that polymorphic primitives can safely work on. This should solve your problem. Yes, in deed! Very nice. As there are still unused bits in the page table... I'm thinking about the following. One bit could be used by the GC to indicate that values somewhere in the page are still referenced from heap memory. At the beginning of the mark phase the bit is cleared. Whenever a pointer is seen that goes to a page marked as In_static_area, the bit for the area is set. As before, the mark phase ignores these pointers otherwise (they are not followed). After the mark phase, one could check which of the pages are still used, and this would make it possible to call finalisers for whole pages. As we only add an else case to the mark loop, the costs for this are negligible. In the shared memory scenario, each process mapping share memory could define such a finaliser, and only when it is called the memory block is unmapped. (And when a global counter is decreased to 0, the shared memory object as such can be deleted. But that's outside the ocaml runtime.) With that help, shared memory would be relatively safe to use. The only remaining trap would be value pointers that go from shared memory to heap memory - but ocaml has already lots of ways to control mutability of values, so I think it is ok to let the programmer do this. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: multicore wish
Am Sonntag, den 27.12.2009, 13:45 +0100 schrieb Goswin von Brederlow: Jon Harrop j...@ffconsultancy.com writes: On Thursday 24 December 2009 13:19:52 Goswin von Brederlow wrote: Jon Harrop j...@ffconsultancy.com writes: No, in OCaml I fork every child. That is the only transparent way to give the child a coherent view of the heap but it is extremely slow (~1ms): So if you add a (sleep 60) to the ocaml code then ocaml gets even more worse than F#? You are comparing an implementation that is specifically optimized to use things F# is good at and ocaml is bad. To give a fair comparison you need to optimize the implementation to the language. Post a better OCaml solution if you can. Each process then has a work queue which is works through. So they will always use the local data. Only when a queue runs dry they steal from another process and ruin locality. You are correctly describing the efficient solution based upon work-stealing queues that F# uses but OCaml cannot express it. You mean that you didn't implement that way. No, I mean OCaml cannot express it. I can easily express that with closures and pre-forked worker threads. OCaml's threads do not run in parallel so that will not work. Or just implement it properly in ocaml. The problem part is the GC. The rest is easy. No kidding. There is one implementation: http://www.algo-prog.info/ocmc/web/ But as said maybe not a verry good one. I tried implementing parallel threads under the original GC by forking multiple instances of the same program and using a Bigarray to mmap /dev/null for shared memory between the instances. That works for sharing primitive types, flat records (records of primitive types) and even fixed (or bound) sized structures but it gets more and more complex to share each and needs some Obj magic, marshaling or C stubs (except for primitive types). It works but is a real hack. It works with all types: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netsys/netsys_mem.mli look for init_value. It's non-released code yet. However, there are some problems: Values outside the heap do not support the polymorphic comparison and hash functions. That's a hard limitation, e.g. you cannot even compare two strings, or build a hash table with strings as keys. That limits the usefulness of shared memory. Also, as we are striving for safe programs, there is the question when shared memory can be safely released. The GC could help here, but there are no ways to hook in, e.g. to get notified when there are no pointers anymore to a value living in shared memory. Gerd I hope someone will pick up the pices and update OCaml4Multicore to the latest ocaml or maybe for ocaml to add it directly. If not then I fear ocaml will be left behind soon. MfG Goswin ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: OCaml is broken
Am Dienstag, den 22.12.2009, 13:04 +0100 schrieb Erik Rigtorp: On Mon, Dec 21, 2009 at 23:50, Erik Rigtorp e...@rigtorp.com wrote: Some IPC Benchmarks, Solaris 10 on a quad core Intel Core2 Duo. The benchmarks are running on a cpuset with 1 core. I measure the time from sending in one process until the other process receives the message. So a context switch and the message passing is included in the measurements. Max/Min/Avg * Pipes: 28205/5973/6259 * Unix domain sockets: 44256/7748/8153 * SYSv message queues: 19197/5895/6173 * Posix message queues: 37399/10965/11303 * TCP on loopback: 29017/7471/7885 So the latency is roughly 10µs for all these solutions. That latency is pretty high and would be several times the processing time of the message itself. Some more benchmarks: Max/Min/Avg * Spinlocking shm: 50897/403/761 (This one utilizes multiple cores, since one core is just burning while waiting for data) * Pthreads mutex shm: 27582/5246/6577 Forgot to say that all measurements are in nanoseconds. That's for communication between processes, right? How would the picture be different (especially comparing the latter two) if you do message passing between threads? If I remember correctly, threads are more light-weight in Solaris than processes. That could also affect context switching times, and scheduler decisions. Do you have source code? I could also run in on Linux, for comparison. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Looking for information regarding use of OCaml in scientific computing and simulation
Have you ever tried writing a significant or complex algo using message passing? Its fun if you have nothing better to to -- its a good intellectual challenge. You can even learn some interesting computer science while you do it. However, if you are interested in merely using the system to do your real work, then writing message-passing code is an utter waste of time -- its difficult, time-consuming, error prone, hard to balance and optimize tune, works well only for embarrasingly parallel code, etc. Even the evil slow-down of NUMA is often better than trying to performance-tune a message-passing system. Well, it is true that message passing is more expensive, and you need bigger data sets until it is worth it (nonsense to do a 10x10 matrix multiplication with message passing). However, I don't think it is that complicated as you describe. Especially ocaml's uniform representation of values can help a lot, and hide many of the low-level details. It could be a lot like continuation-passing style. Hard to balance and optimize tune: This is true for _any_ parallelization strategy. Let me put it this way: suggesting that programmers can write their own message-passing system is kind of like telling them that they can write their own garbage-collection system, or design their own closures, or they can go create their own type system. Of course they can ... and if they wanted to do that, they would be programming in C or assembly, and would probably be designing new languages. Cause by the time you get done with message passing, you've created a significant and rich programming system that resembles a poorly-designed language... been there, done that. See it this way: The typical ocaml programmer doesn't like system programming, and will seldom/never touch C or assembly. The task it to help this kind of programmer, and to make parallel programming available in a higher-level way than it is available elsewhere. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] general question, was Re: OCaml is broken
Am Montag, den 21.12.2009, 15:19 +0100 schrieb Keyan: Hi, i have a large project written in C++, for which i am planing to write add-ons and tools in ocaml, e.g. different tools to analyse my code (dependency stuff), an interpreter for a script-language i plan to include, etc, etc. form my time at the uni i remembered that ocaml allows to compile libraries which can be included in c/c++ program, and i know people who use it extensively in other projects. therefore, i decided to give ocaml a try. i like functional programming, and my first steps with ocaml are very promising. following this discussion, i am not so sure anymore, if ocaml is a good decision. may be i got this discussion wrong, but if ocaml is dying out, i might have to look for another functional programming language to use with my project. please dont take this email as an offence. i am just curious. at this point, i can still easily look for an alternative to ocaml, so its best to ask now. Please don't believe Jon's propaganda. He has just very specific needs (high performance computing on desktops), and generalizes them in the way it's not perfect for me anymore, so it's bad anyway. He has been doing that for years now, not seeing that he really harms the way ocaml is seen by newcomers. The examples you mention are good matches for using ocaml - symbolic programming with lots of terms and trees. That's the stuff ocaml was originally developed for, and it delivers excellence performance. Also, ocaml is still backed by INRIA, and there is still a large community, including a growing number of industrial users. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: OCaml is broken
Am Sonntag, den 20.12.2009, 08:47 -0500 schrieb Yaron Minsky: On Sun, Dec 20, 2009 at 7:21 AM, Erik Rigtorp e...@rigtorp.com wrote: The first step for OCaml would be to be able to run multiple communicating instances of the runtime bound to one core each in one process and have them communicate via lock free queues. We've done some experiments in this direction at Jane Street. On Linux, we've been able to get fast enough IPC channels for our purposes that slamming things into the same memory space has not in the end been necessary. (There is I agree some pain associated with running multiple runtimes in the same process. If you're interested, contact me off-list and I can try to get you some of the details of what we ran into.) But have you tried using shared-memory segments for communicating between different processes? You say the latencies are too high, but do you have any measurements you could share? Have you tried queues using shared memory segments, in particular? Inter-thread communication has latency as well, and the performance issues depend on lots of things, OS and hardware platform included. It would help in understanding the tradeoffs. I'm also experimenting now with shared memory (shm) as fast IPC mechanism. I've extended ocamlnet with a few functions that allow to copy an ocaml value into a shm segment which is accessible as bigarray: https://godirepo.camlcity.org/svn/lib-ocamlnet2/trunk/code/src/netsys/netsys_mem.mli Look especially for init_string. (I've also to mention Ancient here which inspired to this work.) Having ocaml values in shm saves us from some marshalling costs which is right now the biggest performance penalty when using multiple processes. However, this causes some problems, and at some point modifications of the ocaml runtime will be necessary: - The polymorphic equality and hash primitives do not work anymore for values in such shm segments (and that really hurts, especially string comparison is broken) - Given that the shm segment is set to read-only after being set up, it is not possible to have pointers from shm to other memory regions. This is good, as this would be very dangerous (GC may delete or move values in the regular heap). However, the question arises when the shm segment can be deleted. We would need help by the GC to identify segments that are no longer referenced. Without that, shm will be restricted to a role as low-level inter-process buffers. As we go to higher-and-higher numbers of cores, I suspect that message-passing solutions are likely to scale better than shared memory, so I'm not so sure that OCaml is on the wrong path here. I think that most of the work that's needed is going to come in the form of libraries, with only a little work in the compiler and the runtime. Given that, I think this is an issue for the community to solve, not INRIA. Well, message passing and shm do not exclude each other. We should refine the terminology here: Actually, shm is just a basic mechanism where several execution threads (including processes) can share memory. What's often meant is, however, the role it plays for multi-threading, i.e. shared mutable data structures. What's typical here is that several threads write to the same memory regions. I don't know a good name for naming that programming style - maybe multi-threading style shm is the best. I'm working on a local message passing queue that can be used for long messages, based on shm, and where the messages can contain normal ocaml values (although it is likely that these are copied to the normal heap by the receiver, for the above mentioned reasons, but this is an expensive copy). The whole point will be that the data marshalling costs are minimized. So far I can already say, we will need some changes in the runtime to make such a mechanism fast and safe. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] OCaml is broken
Am Samstag, den 19.12.2009, 10:30 +0100 schrieb Erik Rigtorp: Hi! I've been using Erlang and C++ to build a soft real-time system. As the project has evolved we've needed to write more and more of the code in C++ in order to achieve our latency requirements. But C++ is not as performant as you might think until you start to write your own allocators and cache aligning mallocs and datastructures. I've never liked C++ so I decided to try OCaml and built a simple 100 line program to build order books for Nasdaq. Turns out OCaml has really competitive performance while being a really nice language. However OCaml is broken! It does not provide any support for multicore architectures, which by now is considered a bug! It doesn't even allow me to load multiple runtimes into one C program. Please fix OCaml! The first step would be to support multiple runtimes running in the same process communicating using message queues. As you mention order books and soft-realtime, I guess your main concern are minimized latencies. Well, you need then a style of parallelism that focuses on a certain processing path for a single data item, and where the latency is minimized by using several cores. I think ocaml is unsuited for this type of task, but please don't call ocaml broken because of this. Other types of parallelism can be well supported, especially when you can accept multi-processing, and when you focus on larger processing paths and partitioned data sets. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Cannot build OCaml using Cygwin
Am Mittwoch, den 16.12.2009, 17:00 +0100 schrieb Alain Frisch: On 16/12/2009 16:48, Andre Tampubolon wrote: I'm trying to build OCaml 3.11.1 using Cygwin. It's still unsuccessful. $ make -f Makefile.nt world cd byterun ; make -f Makefile.nt all make[1]: Entering directory `/cygdrive/c/cygwin/home/ocaml-3.11.1/byterun' gcc -mno-cygwin -DOCAML_STDLIB_DIR='C:/ocamlmgw/lib' -IC:\cygwin\bin -O -mms -bitfields -Wall -Wno-unused -c interp.c gcc: The -mno-cygwin flag has been removed; use a mingw-targeted cross-compiler. Recent versions of gcc under Cygwin don't support the -mno-cygwin flag. You can still install and use the gcc-3 packages. What I usually do is to copy /bin/gcc-3.exe into $HOME/bin/gcc.exe (and to this directory in front of your PATH). Another way would be gcc -V 3 -mno-cygwin Of course, only for those who don't hardcode gcc arguments... There is also the /etc/alternatives/gcc symlink which can be set to /usr/bin/gcc-3.exe to select the default gcc version. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: OCaml is broken
The following web page describes a commercial machine sold by Azul Systems that has up to 16 54-core CPUs (=864 cores) and 768 GB of memory in a flat SMP configuration: http://www.azulsystems.com/products/compute_appliance.htm As you can see, a GC with shared memory can already scale across dozens of cores and memory access is no more heterogeneous than it was 20 years ago. Also, note that homogeneous memory access is a red herring in this context because it does not undermine the utility of a shared heap on a multicore. The benchmarks they mention can all easily be parallelized - that stuff you can also do with multi-processing. The interesting thing would be an inherent parallel algorithm where the same memory region is accessed by multiple threads. Or at least a numeric program (your examples seem to be mostly from that area). - Have you considered that many Ocaml users prefer a GC that offers maximum single core performance, OCaml's GC is nowhere near offering maximum single core performance. Its uniform data representation renders OCaml many times slower than its competitors for many tasks. For example, filling a 10M float-float hash table is over 18x slower with OCaml than with F#. FFT with a complex number type is 5.5x slower with OCaml than F#. Fibonacci with floats is 3.3x slower with OCaml than my own HLVM project (!). Sure, but these micro benchmarks are first seldom correct, and do not really count for real-world programs. For example, an important parameter of such benchmarks is the frequency the GC runs. Ocaml runs the GC very often - good for latencies, but bad for micro benchmarks because other runtimes simply delay the GC until some limits are exceeded, so these other runtimes often haven't run the GC even once in the short period of time the benchmark runs. It is simply a fact that the ocaml developers had some preferences. E.g. allocating and freeing short-living values is extremely fast (often 10ns). This is very good when you do symbolic computations, or have lots of small strings, but ignorable for numeric stuff, or for programs where the lifetime of allocated memory is bound to server sessions. The minor GC is very fast, but, as you observe, the uniform representation has costs elsewhere. because their application is parallelised via multiple processes communicating via message passing? A circular argument based upon the self-selected group of remaining OCaml users. Today's OCaml users use OCaml despite its shortcomings. If you want to see the impact of OCaml's multicore unfriendliness, consider why the OCaml community has haemorrhaged 50% of its users in only 2 years. Don't see that. That's just speculation - maybe some win32 ocaml users switched to F#, but there are for sure also other reasons than multicore support, e.g. GUIs and better Windows integration. Btw, where do you get your numbers from? There are many, many users for whom multicore is just a useless hype. Either the algorithms are inherently difficult to parallelize (and this is vast majority), or are that easy (like all client/server stuff) that multi-processing is sufficient. You can consider multicore as a marketing trick of the chip industry to let the ordinary desktop user pay for a feature that is mostly interesting for datacenters. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: What is CPAN? (was: Re: [Caml-list] Hoogle for Ocaml)
Am Donnerstag, den 03.12.2009, 21:21 + schrieb ri...@happyleptic.org: (1) A network of redundant mirrors which means you can always get the tarball you need, even when the original site is down: If I understand correctly, GODI site does not store any of the source tarballs, but the makefiles download the sources directly from their respective home, does it ? Can't we use the MASTER_SITE_BACKUP make variables to have one or several backup sites ? There are backups, e.g. http://godi-backup2.camlcity.org/ (3) An excellent search tool: What's wrong with GODI search? (Other than that it isn't comprehensive.) Ok. (4) A central namespace registry for Perl modules. Once someone has the name 'Net::FTPServer', if you want to write an FTP server, you know you need to give it a different name. Ok ; unfortunately no such authority is required to name the few (compared to perl) ocaml libraries. (5) A testing network. (6) A place where you can browse everything that Perl supports: That would be nice to have in Godi as well. (7) A command line tool to download and install CPAN modules: I like godi_console (despite laking fancy colors :-), although I'd like a simpler command line tool to be available as well. You can also use godi_console as command-line tool, try godi_console -help. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Hoogle for Ocaml
Am Donnerstag, den 03.12.2009, 18:45 + schrieb Matthias Görgens: This might be what you're looking for: http://docs.camlcity.org/docs/index.html Thanks. I'll try it. Does it support e.g. searching for (a-b) - [a] - [b] and spitting out List.map? There is no search for type expressions. Here is the query syntax: http://docs.camlcity.org/docs/syntax.html Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: What is CPAN? (was: Re: [Caml-list] Hoogle for Ocaml)
Am Donnerstag, den 03.12.2009, 18:00 + schrieb Richard Jones: On Thu, Dec 03, 2009 at 05:51:00PM +0100, ri...@happyleptic.org wrote: Not really .. I have been meaning for several years to implement something like *CPAN* for OCaml. CPAN is much more than what people here seem to think it is. Out of curiosity, what's in CPAN that's not in GODI ? OK, I knew I'd have to answer this question :-) CPAN is: (1) A network of redundant mirrors which means you can always get the tarball you need, even when the original site is down: http://mirrors.geoexpat.com/cpan/authors/id/R/RW/RWMJ/ http://mirror.unej.ac.id/cpan/authors/id/R/RW/RWMJ/ http://mirrors.ucr.ac.cr/CPAN/authors/id/R/RW/RWMJ/ (more: http://search.cpan.org/mirror) (2) CPAN unpacks each tarball and makes the source and documentation available in a browsable way: http://search.cpan.org/~rwmj/Net-FTPServer-1.122/lib/Net/FTPServer.pm http://cpansearch.perl.org/src/RWMJ/Net-FTPServer-1.122/lib/Net/FTPServer.pm (3) An excellent search tool: http://search.cpan.org/search?query=IO%3A%3Astringymode=all (4) A central namespace registry for Perl modules. Once someone has the name 'Net::FTPServer', if you want to write an FTP server, you know you need to give it a different name. (5) A testing network. When you submit a new version of your module, it is picked up by automated and manual testers around the world, who build and test it on their systems (often oddball ones - eg. I get reports about it failing to build on SunOS and AIX). (6) A place where you can browse everything that Perl supports: http://www.cpan.org/modules/by-module/ http://www.cpan.org/modules/01modules.index.html Which is great advertising for Perl, because you can immediately see the breadth of available libraries for Perl. (7) A command line tool to download and install CPAN modules: http://search.cpan.org/~andk/CPAN-1.9402/lib/CPAN.pm#DESCRIPTION -- Note what CPAN is *not*: It's not a packaging system. Perl has its own packaging standard(s) (like cabal for Haskell), but CPAN doesn't care. It hosts source tarballs. It doesn't store binaries. It's not strongly centralized. Actually, it's very loose indeed. Although there is a central place where you upload modules, they are very loose about naming, content, licensing etc. (within limits of course). -- Out of curiosity, what's in CPAN that's not in GODI ? I think that Gerd Stolpmann has (to his credit) done a lot of the work that CPAN does, but I also think it should be on a firmer footing, mirrored more widely, the search tools should be linked from the OCaml home page, and not tied to building modules, but to hosting source tarballs. And more inclusive - it should include *every* source tarball -- as much OCaml source as possible. Yes, nice goals - but hard to achieve if you only have limited resources. Also, don't forget that Perl attracted a lot of sysadmins, and for them it is a lot of fun to create something like CPAN. For the typical Ocaml developer it is scripting hell. Finally, there is the question of creating normative pressure so sources have some formal uniformity - normal for sysadmins, but a no-go for the creative ocamlists. I'd say it is a different project than GODI to make sources available, browsable and searchable. There's some overlap, though. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] ocamlnet-3.0test1
Hi list, A first testing version of Ocamlnet 3 has been released: Ocamlnet-3.0test1 (download: http://download.camlcity.org/download/ocamlnet-3.0test1.tar.gz) The idea of this release is to make this version available to a larger audience for testing, and to allow everybody to check whether code using this library still works. It is not yet ready for production environments. List of major changes: * Port to Win32 (as outlined in the blog article http://blog.camlcity.org/blog/ocamlnet3_win32.html) * The new Rpc_proxy layer (as described in http://blog.camlcity.org/blog/ocamlnet3_ha.html) * Extensions of Netplex * New implementation of the Shell library for starting subprocesses * Uniform debugging with Netlog.Debug * Exception printers (Netexn) * Introduction of pollsets (Netsys_pollset); removal of Unix.select (i.e. more than 1024 file descriptors) * The netcgi1 library has been dropped in favor of netcgi2 There are also a lot of minor changes. Some of the changes are incompatible with code written for Ocamlnet 2. Testers are especially encouraged to check whether Ocamlnet 3 still works on all platforms, because a lot of new platform-specific code has been added. I mainly tested with Linux and the MinGW port for Win32. The library is not yet available via GODI. I'm working on this. More blog postings will follow describing the highlights. Please report results to g...@gerd-stolpmann.de Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: How to read different ints from a Bigarray?
Am Donnerstag, den 29.10.2009, 21:40 +0100 schrieb Florian Weimer: * Goswin von Brederlow: - The data is passed to libaio and needs to be kept alive and unmoved as long as libaio knows it. It also has to be aligned to a 512-byte boundary, so you can use O_DIRECT. Linux does not support truely asynchronous I/O without O_DIRECT AFAIK, which rarely makes it worth the trouble. Right. There is also the question whether aio for regular files (i.e. files backed by page cache) is continued to be supported at all - it is well known that Linus Torvalds doesn't like it. It can happen that at some day aio will be restricted to block devices only. So I wouldn't use it for production code, but it is of course still an interesting interface. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: How to read different ints from a Bigarray?
Am Freitag, den 30.10.2009, 00:43 +0100 schrieb Goswin von Brederlow: Gerd Stolpmann g...@gerd-stolpmann.de writes: Am Donnerstag, den 29.10.2009, 21:40 +0100 schrieb Florian Weimer: * Goswin von Brederlow: - The data is passed to libaio and needs to be kept alive and unmoved as long as libaio knows it. It also has to be aligned to a 512-byte boundary, so you can use O_DIRECT. Linux does not support truely asynchronous I/O without O_DIRECT AFAIK, which rarely makes it worth the trouble. Right. There is also the question whether aio for regular files (i.e. files backed by page cache) is continued to be supported at all - it is well known that Linus Torvalds doesn't like it. It can happen that at some day aio will be restricted to block devices only. So I wouldn't use it for production code, but it is of course still an interesting interface. Gerd Damn. That seems so stupid. Then writing asynchronous will only be possible with creating a pot full of worker thread, each one writing one chunk. So you get all those chunks in random order submitted to the kernel, the kernel has to reorder them, fit them back together, write them and then wake up the right thread for each piece completed. So much extra work while libaio has all the data already in perfect structures for the kernel. Well, this is exactly the implementation of the POSIX aio functions in glibc. They are mapped to a bunch of threads. And how will you do barriers when writing with threads? Wait for all threads to complete every time you hit a barrier and thereby stalling the pipeline? You can't implement barriers. When you have page-cache backed I/O (i.e. non-direct I/O, no matter of aio or sync I/O) there is no control when data is written. Ok, there is fsync but this is very coarse-grained control. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] How to read different ints from a Bigarray?
Am Mittwoch, den 28.10.2009, 14:54 +0100 schrieb Goswin von Brederlow:
Hi,
I'm working on binding s for linux libaio library (asynchron IO) with
a sharp eye on efficiency. That means no copying must be done on the
data, which in turn means I can not use string as buffer type.
The best type for this seems to be a (int, int8_unsigned_elt,
c_layout) Bigarray.Array1.t. So far so good.
Now I define helper functions:
let get_uint8 buf off = buf.{off}
let set_uint8 buf off x = buf.{off} - x
But I want more:
get/set_int8 - do I use Obj.magic to convert to int8_signed_elt?
And endian correcting access for larger ints:
get/set_big_uint16
get/set_big_int16
get/set_little_uint16
get/set_little_int16
get/set_big_uint24
...
get/set_little_int56
get/set_big_int64
get/set_little_int64
What is the best way there? For uintXX I can get_uint8 each byte and
shift and add them together. But that feels inefficient as each access
will range check and the shifting generates a lot of code while cpus
can usualy endian correct an int more elegantly.
Is it worth the overhead of calling a C function to write optimized
stubs for this?
And last:
get/set_string, blit_from/to_string
Do I create a string where needed and then loop over every char
calling s.(i) - char_of_int buf.{off+i}? Or better a C function using
memcpy?
What do you think?
A C call is too expensive for a single int (and ocamlopt). The runtime
needs to fix the stack and make it look C-compatible before it can do
the call. Maybe it's ok for an int64.
Can you ensure that you only access the int's at word boundaries? If so,
it would be an option to wrap the same malloc'ed block of memory with
several bigarrays, e.g. you use an (int, int8_unsigned_elt, c_layout)
Bigarray.Array1.t when you access on byte level, but an (int32,
int32_unsigned_elt, c_layout) Bigarray.Array1.t when you access on int32
level, but both bigarrays would point to the same block and share data.
This is trivial to do from C, just create several wrappers for the same
memory.
The nice thing about bigarrays is that the compiler can emit assembly
instructions for accessing them. Much faster than picking bytes and
reconstructing the int's on the caml side. However, if you cannot ensure
aligned int's the latter is probably unavoidable.
Btw, I would be interested in your aio bindings if you do them as open
source project.
Gerd
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] threads, signals, and timeout
: Hi, I would like to create different threads where each thread do some computation and are subject to different timeout. Without threads I usually use Unix.alarm with a SIGALARM handler that just raise a Timeout exception and everything works fine, but when I try to do something similar with threads it does not work because apparently the Unix.alarm done in one thread override the Unix.alarm done in another thread. I had a look at thread.mli but was not able to find anything related to timeout. Is there a way to have multiple timeout and multiple threads at the same time ? Here is a program that unforunately get the first timeout, but not the second :( (* ocamlc -g -thread unix.cma threads.cma signals_and_threads.ml *) exception Timeout let mytid () = let t = Thread.self () in let i = Thread.id t in i let set_timeout () = Sys.set_signal Sys.sigalrm (Sys.Signal_handle (fun _ - prerr_endline Time is up!; print_string (Printf.sprintf id: %d\n (mytid())); raise Timeout )); ignore(Unix.alarm 1); () let main = let t1 = Thread.create (fun () - set_timeout (); print_string (Printf.sprintf t1 id: %d\n (mytid())); let xs = [1;2;3] in while(true) do let _ = List.map (fun x - x + 1) xs in () done; () ) () in let t2 = Thread.create (fun () - set_timeout (); print_string (Printf.sprintf t2 id: %d\n (mytid())); let xs = [1;2;3] in while(true) do let _ = List.map (fun x - x + 1) xs in () done; () ) () in Thread.join t1; Thread.join t2; () -- Here is the output Time is up! t2 id: 2 t1 id: 1 id: 1 Thread 1 killed on uncaught exception Signals_and_threads.Timeout the program loops, meaning the second thread never received its timeout ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Dynlink and ocamlfind for camlp4 3.11?
Am Sonntag, den 04.10.2009, 20:53 +0100 schrieb Conglun Yao: In that case, try listing dynlink explicitly, like: ocamlfind ocamlc dynlink.cma -package json-static [etc] $ ocamlfind ocamlc dynlink.cma -package json-static -syntax camlp4o -c test.ml $ ocamlfind ocamlc -I /home/conglun/godi/lib/ocaml/pkg-lib/camlp4 dynlink.cma -package json-static -syntax camlp4o -c t.ml Error: Error while linking /home/conglun/godi/lib/ocaml/std-lib/camlp4/camlp4lib.cma(Camlp4): Reference to undefined global `Dynlink' Anyhow, it sounds like a bug in the META files that are supplied with your cygwin OCaml distribution. I think you are right, the problem happens on the camlp4o or cygwin, which can't load dynlink. But I can't find problem in the camlp4's META (in the attachment) While cygwin cannot load shared libraries from bytecode, it still can load pure bytecode dynamically. It is reasonable that there is a dynlink.cma for this case, and that camlp4lib.cma depends on it. findlib has a special mode for platforms that cannot load shared libraries dynamically. This mode seems to be broken - it calls a script safe_camlp4 instead of camlp4, and safe_camlp4 produces ad hoc a new camlp4-type executable that includes the required C libraries - without that trick you couldn't use json-static at all on these platforms: https://godirepo.camlcity.org/svn/lib-findlib/trunk/tools/safe_camlp4 The question is now whether the error is in this script (does it have to add dynlink.cma anyway?) or whether mkcamlp4 is broken. Can you try to change safe_camlp4 so the line mkcamlp4 -o $tmp_camlp4 $cp4_mods || exit reads mkcamlp4 -o $tmp_camlp4 dynlink.cma $cp4_mods || exit ? (I don't have a cygwin ocaml at hand.) safe_camlp4 should be in the bin/ directory. Gerd -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Missing the cmi files required to use toplevellib.cma
GODI installs the required cmi's. One can easily compile your utility with ocamlc -o cmidump -I /opt/godi-3.11/lib/ocaml/compiler-lib/ toplevellib.cma cmidump.ml For docs.camlcity.org I converted all cmi's to a readable format by a little script that does apply ocamlc -i on the program include M where M is the module to print. That gives finally a nice view on all information, like in http://docs.camlcity.org/docs/godipkg/3.11/godi-lwt/lib/ocaml/pkg-lib/lwt/lwt.cmi_pretty You can switch between the pretty-printed cmi, the mli, and the source files. Ideal for exploring software. Gerd Am Donnerstag, den 01.10.2009, 14:09 +0200 schrieb ri...@happyleptic.org: While learning OCaml, I just coded a small program that dumps the full content of a cmi file. I find this more usefull than ocamlbrowser or to use the toplevel to have a small command line driven dumper, and it was also a good pretext to have a look under the cover. The problem is : most of the usefull types and functions are installed in the toplevellib.cma but I can't use this without the proper cmi files (I need config.cmi for cmi_magic_number, printtyp.cmi and typemod.cmi for printing signatures, but env.cmi would be nice to have as well for read_signature). Of course I can use those left in ocaml-3.11.1 directory after compilation, but having them installed would help the creation and distribution of such tools. But maybe there is an other way to use toplevellib.cma that I'm unaware of ? If not, then why not install these cmi files along with toplevellib ? Anyway, if anyone is interrested the tool is attached. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] xpath or alternatives
Am Mittwoch, den 30.09.2009, 15:39 +0200 schrieb Stefano Zacchiroli:
On Mon, Sep 28, 2009 at 01:17:45PM +0100, Richard Jones wrote:
I need to do some relatively simple extraction of fields from an XML
document. In Perl I would use xpath, very specifically if $xml was an
XML document[1] stored as a string, then:
my $p = XML::XPath-new (xml = $xml);
my @disks = $p-findnodes ('//devices/disk/source/@dev');
push (@disks, $p-findnodes ('//devices/disk/source/@file'));
I've just realized that this thread can look a bit ridiculous, at least
for people used to other languages where XPath implementations can even
be found in the language standard library (the best solutions we have
thus far are: a 40-line xml-light solution, the need to use a modified
version of the OCaml compiler [yes, I know, it is compatible, but still
...], Galax with unreachable homepage, ...).
So, I was wondering, has anybody ever tried to develop an XPath
implementation on top of, say, PXP? The original announcement page of
PXP (now archived) mentions rumors about people which, back then, were
developing it. Has anything ever been released?
No. However, there is a little XPath evaluator in SVN:
https://godirepo.camlcity.org/svn/lib-pxp/trunk/src/pxp-engine/pxp_xpath.ml
I have never found the time to complete it, and to add some syntax
extension for painless use. But maybe somebody wants to take this over?
Gerd
At first glance, it doesn't seem to exist any specific typing problem,
at least with XPath 1.0, since the PXP node interface is already common
for all node types. Sure XPath 2.0, when static typing is in use, can be
better integrated with the language, but that's probably already
happening in Galax.
[ Cc-ing the PXP mailing list ]
Cheers.
--
Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany
g...@gerd-stolpmann.de http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Incremental linking
Am Dienstag, den 29.09.2009, 20:39 +0200 schrieb Dawid Toton: I have lot of modules and they are compiled to native code. So I have .cmx and .o files and want to link them faster. Well, you can link several .cmx files (and their accompanying .o files) to a .cmxa file (and an accompanying .a file): ocamlopt -a You cannot do the same again on the next level, i.e. link several .cmxa/.a together to get another .cmxa/.a. (I don't remember why this restriction exists.) Gerd Is is possible to make linking an associative operation acting on modules? I would like to do something like the following: Knowing the correct partial order of modules (that compiler requires) I can create a tree that preserves that order. Leafs are modules. Other nodes of the tree correspond to a result of linking all descendant modules. Modules that are frequently recompiled are placed closer to the root. This way I expect to execute less linking operations during development. Documentation of ld says that files produced with --relocatable can be used as intermediate partially linked files. Can something like this be done with object code produced by ocamlopt? I don't know ocaml-specific details of linking, so maybe I overlook some obvoius obstacle? Dawid ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] JIT HLVM, LLVM
Am Sonntag, den 27.09.2009, 13:35 -0400 schrieb Edgar Friendly: Vincent Aravantinos wrote: Hi, I think what Jon means is that, with JIT, polymorphic functions can be specialized at run-time and allow optimizations that are not currently achieved by the Ocaml native code compiler. V. The alternative to specializing at runtime using JIT is to do it at compile time (/ link time) using a form of whole-program analysis. How expensive would this be, and how hard would it be to still support separate compilation? And how much would the OCaml world cry if we didn't have fully-separate compilation? We don't have fully separate compilation anyway (in native mode). The compiler can already dump an intermediate representation of functions into .cmx files, and can use that for cross-module inlining. This feature is optional for the user. Of course, it increases the number of modules that need to be recompiled when something is changed. If we keep that spirit of giving the user a choice: Of course the Ocaml world would appreciate when the actual code generation can be delayed in order to gain performance improvements by whole-program analysis. However, for projects with several 100KLOC the compile time could be drastically increased, and I don't think users with such large programs would actually use that feature. Gerd At the moment, and for the foreseeable future, anything binary is bound tightly to the compiler version, and binary distribution of modules seems pretty much impossible due to this and unknown other factors. How much easier would the above be given the source code / some intermediate representation to generate specialized code with? E ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Cache algorithms: implementation or library available?
Maybe the Wink cache server is interesting for you: http://oss.wink.com/cache/ It is an LRU cache with a maximum size, and entries can also expire by time. If you are more looking for a local cache module, you can pick the relevant parts out of the cache_server.ml implementation. Gerd Am Dienstag, den 22.09.2009, 11:29 +0100 schrieb Hugo Ferreira: Hello, I would like to know if anyone has or knows of an Ocaml library or open-source code implementation of some cache algorithms (example: least recently used). Basically I need to cache a function that maps a list of ordered integers into a value (float, integer). I would like something that allows setting a maximum size of the map and automatically discards the data. Any pointers are appreciated. TIA, Hugo F. ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] PXP error messages
Am Donnerstag, den 03.09.2009, 10:30 +0200 schrieb Florian Hars: I think that Fatal error: exception Pxp_core_types.A.At(In entity [toplevel] = PRIVATE, at line 2, position 175: , _) is a suboptimal message for No DTD found, maybe you should use Pxp_tree_parser.parse_wfdocument_entity instead RTFM. How to get readable exceptions: http://projects.camlcity.org/projects/dl/pxp-1.2.1/doc/manual/html/ref/Intro_getting_started.html#exn Gerd - Florian ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann, Bad Nauheimer Str.3, 64289 Darmstadt,Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: netclient-ssl
Am Mittwoch, den 15.07.2009, 18:09 -0700 schrieb Dave Benjamin: Hi Mykola, Does anybody can show an example of using Gerd Stolpman's netclient- ssl? I'm especially interested in performing https call. netclient-ssl? What's that? (Rhetoric question - such a library does not exist.) I think you posted this to fa.caml using Google Groups. Unfortunately, fa.caml is (or should be) a read-only copy of the messages on the caml-list mailing list. I have copied this to the list so that others may respond. You may wish to sign up to continue this conversation: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list To answer your question, I don't think it is very easy to do HTTPS using netclient-ssl - you have to write your own version of the HTTP protocol because Ocamlnet's HTTP and SSL support are not compatible with each other. Yes. Unfortunately, netclient uses an older approach for doing socket I/O where one cannot plug in SSL support. I have right now no time for doing the necessary changes (unless somebody would pay me). Gerd You will probably have a much easier time using Curl: http://sourceforge.net/projects/ocurl/ Good luck, Dave -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Virtual packages in findlib
Am Mittwoch, den 08.07.2009, 03:35 -0700 schrieb Dario Teixeira: Hi, Can you explain that? I don't see why virtual packages can resolve ordering problems. Because without virtual packages, one cannot list the true dependencies of a package, thus causing ordering problems. Consider the minimal example of there being two packages: A and B, with B depending on A, and therefore having to be linked after. However, A is not a real package, but instead a virtual one instantiated by either A1, A2, A3, etc. But alas, there is no virtual package support in findlib (I assume). Because of this, the META file for B does not list A as a dependency. (Yes, the META is wrong, but how could it be made right?) Now, we tell ocamlbuild that the project depends on A1 (for instance) and B. Because there is no *declared* dependency between A1 and B (though in fact there should be), ocamlbuild probably assumes that it is indifferent to link A1; B or B; A1. However, the latter case will cause an error. Therefore, there are two solutions to this problem: a) Make ocamlbuild preserve the order of the given packages *when there is no explicit dependency among them*. (Obviously if there are dependencies then Ocamlbuild should re-order the packages as fit). b) Find some virtual package mechanism in findlib (thus rendering the hack a) moot). Well, you can make dependencies conditional using the predicate feature. In B you can write: requires(P_A1) = A1 requires(P_A2) = A2 etc. and when linking you have to select which dependency to take by giving the right -predicates switch: e.g. -predicates P_A1. Also, there are subpackages (useful when you want to put A1 and A2 into the same directory). So you can create a fancy package A with subpackages A1 and A2 like: requires(P_A1) = A.A1 requires(P_A2) = A.A2 requires = A.A1(* default *) package A1 ( (* definitions for A1 *) requires = ... archive(byte) = ... archive(native) = ... ) package A2 ( (* definitions for A2 *) requires = ... archive(byte) = ... archive(native) = ... ) And B simply lists A as dep, and A is dependent on either A.A1 or A.A2, as given by predicates. This is not really a virtual package as you describe it. It was developed to support several versions of the same package, e.g. one version with enabled debugging, and the other without. The predicates work more or less orthogonal to the dependency graph. Gerd Cheers, Dario Teixeira ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Strange buffering interaction: stdout / file_descr
This is the somewhat expected behavior, although a bit surprising at the first glance. create_process isn't an atomic operation. Actually, it forks the process (and thus implicitly duplicates the stdout buffer). In the forked child it tries to exec the new executable. If this fails, it does a regular exit(), and the buffers are flushed. One can argue that this is wrong - the implementation should better do an _exit (i.e. immediately exit the child process without doing any sort of cleanup). AFAIK, system() implementations do this. The standard way of checking for exec errors is to test the exit code, and if it is 127, exec or the code immediately before exec failed. That's a convention only, i.e. nothing enforces this. Gerd Am Samstag, den 23.05.2009, 16:17 -0500 schrieb Alexander Fuchs: let () = print_string main; let (stdout_r, stdout_w) = Unix.socketpair Unix.PF_UNIX Unix.SOCK_STREAM 0 in let (stdin_r, stdin_w) = Unix.socketpair Unix.PF_UNIX Unix.SOCK_STREAM 0 in let cmd = none in (* without flush reading from stdout_r yields main *) (* flush stdout; *) let pid = Unix.create_process cmd [| cmd; -version |] stdin_r stdout_w stdout_w in Unix.close stdin_r; Unix.close stdout_w; begin let (outs, _, _) = Unix.select [stdout_r] [] [] (-1.0) in match outs with | [out] - let buffer = String.make 100 ' ' in let size = Unix.read out buffer 0 100 in if size 0 then begin print_endline (buffer) end else begin print_endline create_process failed end | _ - assert false; end; Unix.close stdout_r; Unix.close stdin_w; Unix.kill pid Sys.sigkill; let (_, _status) = Unix.waitpid [] pid in () -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] ocamlfind question on cygwin
Am Mittwoch, den 06.05.2009, 19:23 +0200 schrieb Alan Schmitt: On 6 mai 09, at 18:11, Dmitry Bely wrote: On Wed, May 6, 2009 at 7:57 PM, Alan Schmitt alan.schm...@polytechnique.org wrote: Hello, I'm trying to debug some installation issue on godi/mingw, and it seems to be a problem with ocamlfind using a cygwin console. When I try: $ ocamlfind install lwt -destdir /home/Administrateur/godi/lib/ocaml/pkg-lib META ocamlfind: Bad configuration: Cannot mkdir /home/Administrateur/godi/lib/ocaml/pkg-lib\lwt because a path component does not exist or is not a directory Probably your ocamlfind is a mingw (native Win32) application, and so you cannot use cygwin paths (/home/...). As godi uses cygwin paths, does it mean that one cannot use ocamlfind in godi packages for mingw? Actually, this is a big problem. In godi_console, I worked around by translating paths between cygwin and Win32 as needed. All other godi apps do not provide this convenience feature - they are win32 apps and require win32 paths. In order to help packages, there is LOCALBASE_NATIVE. It is LOCALBASE translated to Win32 style. Look into the godi-findlib package for an example how to use it. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] Ann: Hydro-0.7
Hi list, Hydro 0.7 is released. Hydro is an independent implementation of the ICE middleware (originally by ZeroC, see zeroc.com). It is now a bit more complete: * The hydrogen generator supports a faster way of doing the language mapping (to be enabled with the -dc switch), now bypassing the value tree representation in most cases. * It is possible to register Hydro servers at an ICE registry. * Support for stringified proxies * Hydromon for checking the lifeliness of remote servers Additionally, there are many smaller changes and bugfixes. Get Hydro at oss.wink.com/hydro. There is also a GODI package for it. Hydro is a development effort by Mylife.com (formerly Wink Technologies). Hydro is used in the people search component to connect the various server with each other. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Building pcre-ocaml on OCaml 3.11.0 on MinGW
Am Donnerstag, den 23.04.2009, 11:30 +0200 schrieb Alain Frisch: Hi David, David Allsopp wrote: I've just had an enlightening few hours getting pcre-ocaml to compile under Windows ... The main thing that's got me puzzled is the renaming of libpcre.dll.a and libpcre.a that I have to do to get the thing to link. Thanks for investigating this issue! The current SVN version of flexlink prefers XXX.dll.a over XXX.a. We will see whether it solves more problems than it creates... Of course, it is always possible to pass explicitly the complete file name instead of -lXXX. To build and use ocaml-pcre, I've had to remove the -I /usr/include from this line in OCamlMakefile: CFLAGS_WIN32 := -mno-cygwin -I /usr/include Otherwise, Cygwin's headers are used instead of mingw's ones (in /usr/include/mingw), and we get a dependency e.g. to the Cygwin symbol _ctype_ which is not available on mingw. Have you had to do something similar? I can confirm this. I also got an error about _ctype_, and could solve this by removing -I /usr/include. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Ocamlfind query
Am Donnerstag, den 23.04.2009, 08:41 -0700 schrieb Dario Teixeira: Hi, I would like to get the name of the cma associated a findlib package. I expect this to be available via the query command, but running ocamlfind query -l pkg_name always lists the archive as empty. Is there some other option? Say -predicates byte or -predicates native Gerd Best regards, Dario Teixeira ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] netplex multi-thread asynchronous processor for passive clients
Am Freitag, den 03.04.2009, 17:19 +0600 schrieb Serge Sivkov: Hello, I want to convert my synchronious multi process netplex service to asynchronous multi thread one. I don't understood how can i send data from #receive_message to #process for asynchronous worker (ideally more than one per thread). Here is my current code : class my_hooks = ... method receive_admin_message container name args = let s = admin message to contaner: ^ name in container#log `Info s; let aux (proto,fds) = Array.iter (fun fd - container#socket_service#processor#process (fun () - ()) container fd proto) fds in List.iter aux container#socket_service#sockets end class ts_alfa_processor hooks : Netplex_types.processor = ... method process ~when_done container fd proto_name = let s = sprintf process called with %d and %s (int_of_file_descr fd) proto_name in container#log `Info s; let ch = Unix.out_channel_of_descr fd in let rec aux () = let s = (Sexp.to_string (sexp_of_msg dfl_msg)) ^ \n in output_string ch s; flush ch in (*+aux() for synchronious version *) aux () method supported_ptypes = [ `Multi_processing; `Multi_threading ] end That code doesn't work because method process called from receive_admin_message got wrong fd as argument (i assume, that's parent fd used to accept right one). No, that does not work. process must be called from the right environment. What is the right way to write porcessor which must: - work asyncroniously with multithread workload manager (ideally, many jobs per thread) - work with clients whom don't send any packets - get data to send from messages sent by other service e.g. open a Unix domain socket, and send data over that socket. Right now, there is no faster way that is officially supported. For convenience, you can define RPC procedures that do all the serialization details. What could work if sender and receiver are on the same event system: The sender puts a special event with the message onto the event queue, and the receiver installs an event handler listening for such events. I don't have demo code for something like this, however. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] XML output
Am Dienstag, den 17.03.2009, 12:01 +0100 schrieb Rémi Dewitte: Hello, I have used pxp to parse xml and I am happy with it. I'd like now to produce xml and wonder what are the options to do so (possibly the simpliest). Maybe not the simplest: Use the PXP preprocessor to create the output tree, and print the tree: http://projects.camlcity.org/projects/dl/pxp-1.2.1/doc/manual/html/ref/Intro_preprocessor.html http://projects.camlcity.org/projects/dl/pxp-1.2.1/doc/manual/html/ref/Pxp_document.document.html#2_WritingdocumentsasXMLtext I think I am going to start with the Printf module. I wonder how well it handles utf8 for example. UTF-8 are just bytes for printf. And I'll have to write a kind of xml_encode function. I am pretty sure it has already be done somewhere ! let xml_encode = Netencoding.Html.encode ~in_enc:`Enc_utf8 ~out_enc:`Enc_usascii ~prefer_names:false () That would assume the input is UTF-8 encoded, and the output is ASCII-encoded. You can control which ASCII characters get the special XML representation ...; with the unsafe_chars optional argument. Docs are at http://projects.camlcity.org/projects/dl/ocamlnet-2.2.9/doc/html-main/Netencoding.Html.html Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
[Caml-list] PXP-1.2.1 released
Hi, after 4 years the next stable version of PXP is released. Please have a look at http://projects.camlcity.org/projects/pxp.html for download links and the new PXP Reference manual. This release focuses on documentation. The mentioned manual has been written from scratch, and PXP comes now with ocamldoc-generated HTML pages. There are also a few changes in the code (see the README). GODI packages have been updated. A lot of fun for the next 4 years, Gerd P.S. I've written a bit about this version in http://blog.camlcity.org/blog/pxp121.html -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany g...@gerd-stolpmann.de http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Serialisation of PXP DTDs
Am Mittwoch, den 22.10.2008, 13:11 -0700 schrieb Dario Teixeira: Hi, I am using PXP to parse the MathML2 DTD. This is a fairly large DTD, which even on a fast machine takes several seconds to parse. I am therefore looking at ways to serialise a parsed DTD, in a such a way that it can be reused by other processes. Does PXP already offer primitives for (un)serialising DTDs? (I couldn't find any). Note that using Marshal is out of the question, because DTDs are stored as objects, and we all know that objects cannot be serialised across process boundaries. But are there alternative solutions I'm overlooking? No, there is currently no built-in function to serialize DTD's. The DTD objects are, however, mostly containers, and you can get all their properties by invoking methods of the object interface. That allows it to do your own serialization. You are a bit dependent on the PXP version then, but I don't think the interface of DTD's will change anytime soon. Gerd On a more general but related note, I think we should start an OSP discussion about standardising serialisation methods. The rationale should be obvious. Myself, I am partial to Sexplib, since it is reasonably fast, very simple to use, human-readable, and future-proof. I reckon that bin-prot could also be considered, as long as at some point the binary format is set in stone, or at least deserialisers are always backwards compatible. Any other opinions? Thanks for your time! Cheers, Dario Teixeira ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Serialisation of PXP DTDs
Am Donnerstag, den 23.10.2008, 23:05 +0200 schrieb Mauricio Fernandez:
I have been working for a while on a self-describing, compact, extensible
binary protocol, along with an OCaml implementation which I intent to release
in not too long.
It differs from sexplib and that bin-prot in two main ways:
* the data model is deliberately more limited, as the format is meant to be
de/encodable in multiple languages.
* it is extensible at several levels, achieving both forward and backward
compatibility across changes in the data type
You can think of it as an extensible Protocol Buffers[1] with a richer data
model (albeit not in 1:1 accordance with OCaml's for the above mentioned
reason).
Have you looked at ICEP (see zeroc.com)? It has bindings for many
languages, even for Ocaml (http://oss.wink.com/hydro/).
It is, however, not self-describing. Anyway, you may find there ideas
for portability.
Gerd
In the criteria you gave in another message, namely
(1) ease of use
(2) future-proofness
(3) portability
(4) human-readability,
it does fairly well at the 3 first ones --- especially at (2) and (3), which
were poorly supported by existing solutions (I looked into bin-prot, sexplib,
Google's Protocol Buffers, Thrift and XDR; I also referred to IIOP and ITU-T
X.690 DER during the design). Being a binary format, it obviously doesn't do
that well at (4), but it is possible to get a human-readable dump of the
binary data even in the absence of the interface definition, making
reverse-engineering no harder than sexplib (and arguably easier in some ways).
For example, here's a bogus message definition to illustrate (2) and (4).
This protocol definition is fed to the compiler, which generates the OCaml
type definitions, as well as the encoders/decoders and pretty-printers (as you
can see, the specification uses a mix of OCaml, Haskell and C++ syntax, but
it's pretty clear IMO)
type sum_type 'a 'b 'c = A 'a | B 'b | C 'c
message complex_rtt =
A {
a1 : [(int * [|bool|])];
a2 : [ sum_typeint, string, long ]
}
| B {
b1 : bool;
b2 : (string * [int])
}
The protocol is extensible in the sense that you can add new constructors to a
sum or message type, add new elements to a tuple, and replace any primitive
type by a sum type including the original type. For instance, if at some point
in time we find that the b1 field should have a different type, we can do
type bool_or_something 'a = Orig unboxed_bool | New_constructor 'a
and then
...
| B { b1 : bool_or_somethingsome_type; ... }
This, along with a way to specify default values, allows both forward and
backward compatibility.
The compiler generates a pretty printer for these structures, useful for
debugging. Here's a message generated randomly:
{
Complex_rtt.a1 =
[ ((-5378), [| false; false; false; true; true |]);
(3942717140522000971, [| false; true; true; true; false |]);
((-6535386320450295), [| false |]); ((-238860767206), [| |]);
(1810196202, [| false; false; true; true |]) ];
Complex_rtt.a2 =
[ Sum_type.A (-13830); Sum_type.A 369334576; Sum_type.A 83;
Sum_type.A (-3746796577167465774); Sum_type.A (-1602586945) ] }
Now, this is the information decoded in the absence of the above definitions
(iow., what you'd have to work with if you were reverse-engineering the
protocol):
T0 {
T0 [
T0 { Vint_t0 (-5378);
T0 [ Vint_t0 0; Vint_t0 0; Vint_t0 0; Vint_t0 (-1);
Vint_t0 (-1)]};
T0 { Vint_t0 3942717140522000971;
T0 [ Vint_t0 0; Vint_t0 (-1); Vint_t0 (-1); Vint_t0 (-1);
Vint_t0 0]};
T0 { Vint_t0 (-6535386320450295); T0 [ Vint_t0 0]};
T0 { Vint_t0 (-238860767206); T0 [ ]};
T0 { Vint_t0 1810196202;
T0 [ Vint_t0 0; Vint_t0 0; Vint_t0 (-1); Vint_t0 (-1)]}];
T0 [ T0 { Vint_t0 (-13830)}; T0 { Vint_t0 369334576}; T0 { Vint_t0 83};
T0 { Vint_t0 (-3746796577167465774)}; T0 { Vint_t0 (-1602586945)}]}
(I'm still changing some details so it might look better than this shortly.)
It's not a drop-in solution like sexplib's with sexp, by design (since it is
meant to allow interoperability between different languages), but it's still
fairly easy to use.
If you're interested in this, tell me and I'll let you know when it's ready
for
serious usage.
[1] http://code.google.com/p/protobuf/
--
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
[EMAIL PROTECTED] http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml
Re: [Caml-list] XML library for validating MathML
Am Donnerstag, den 18.09.2008, 10:12 +0100 schrieb Till Varoquaux: PXP is tough to work with and feels a bit crazy but it is good with standards (It can sort out any DTD's I have ever thrown at it). xml-light is, well, very broken (it doesn't even support charcode switching). There are several XML parsers in OCaml and I've had a stint with a few of them; the only two I would consider using are expat and Pxp with a marked preference for the later. PXP can be very confusing and feels over engineered at times but it does the job. And remember parsing XML is a hard job, much harder than we often give it credit for Hats off to Gerd for providing us with a proper parser. Thanks. Initially, I thought XML is an easy format - because it looks easy. But the specs are really challenging - full of bad compromises, and I would expect that a widely adopted standard has to undergo some evaluation of its practicability before it is published. For instance, there are very strict rules where whitespace has to be in XML, and where it must not occur. E.g. tag x=ay=b is considered as illegal because of the missing space between the attributes. The whitespace rules make it practically impossible to use a yacc-generated parser (my first attempt was ocamlyacc-based, and it sort of worked after implementing lots of parsing tricks, but it was impossible to fix all errors, although the XML grammar is quite short after all). There are further complications in the XML standard, and after all, it is very difficult to implement it even on the most basic level. So there are many parsers now out there that do not do that, but rather implement a subset because this is easier and parsing is faster. There is much more to say about shortcomings in XML, or the XML standardization process. It is now an unnecessary complicated technology. I would advise everybody to use it only when there is no way around it, e.g. for exchange of structured data between organizations. I've got now a few hours of sponsorship for PXP. I'll try to improve the documentation, because there are some parts that need more explanation (where people feel it is over-engineered, but as Vincent pointed out, it's the standard that demands it). Gerd Till On Thu, Sep 18, 2008 at 9:38 AM, Vincent Hanquez [EMAIL PROTECTED] wrote: On Wed, Sep 17, 2008 at 11:58:05AM -0700, Dario Teixeira wrote: Given a string containing a mathematical expression in the MathML markup, I need to verify that the expression is indeed valid MathML. I am therefore looking for an XML library that can verify an expression against a given DTD. Now, I have tried Xml-light, and the code I used is listed below. Unfortunately, it fails when trying to parse MathML's DTD (it's the standard DTD from the W3C). I have tried simpler DTDs, and it does work with them; am I therefore correct in assuming that Xml-light can only handle a particular version/subset of DTD features? I don't know about validation (i'll probably suggest looking at PXP tho), but xml-light is very bad for XML compliance. the library is (happily) parsing XML files that it shouldn't, which tell a lots concerning its validation abilities ... for example, the XML supported character range is not even checked: Xml 1.0 specification -- 2.2 Characters Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10] others problems include (uncomplete list): - complete unicode un-awareness - funny wrong entities handling -- Vincent ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] XML library for validating MathML
Am Donnerstag, den 18.09.2008, 10:58 -0700 schrieb Dario Teixeira:
Hi,
Well, as it turns out, building a basic Hello World in PXP is relatively
simple (I followed the manual which is very helpful in the beginning).
However, though the DTD validation works fine with the simple examples I
tried,
it fails for a MathML document. Note that I am using the DTD as provided
by the W3C, available from here: http://www.w3.org/Math/DTD/mathml2.tgz
When processing the MathML DTD, PXP outputs a few a warnings about entities
declared twice, about names reserved for future extensions, and quite a
lot of warnings about code points that cannot be represented. I can ignore
those for now.
Code points: Note that PXP defaults to ISO-8859-1 as character set. Use
it in UTF-8 mode to get rid of these warnings.
When it does fail, this is the error produced:
In entity ent-isonum = PUBLIC -//W3C//ENTITIES Numeric and Special Graphic
for MathML 2.0//EN isonum.ent, at line 28, position 44:
Called from entity [dtd] = SYSTEM mathml2.dtd, line 1969, position 0:
ERROR (Well-formedness constraint): The character '' must be written as
'amp;'
Looking at the isonum.ent file (packaged with the W3C zip), these are
the contents of line 28, where the error occurs:
!ENTITY amp #x26;#x00026; !--=ampersand --
Well, the inner entities are again expanded when an entity is expanded.
The correct way to define amp; is
!ENTITY amp #x26;#x26;
i.e. no second . At _definition_ time this gives #x26; (the first
#x26; is expanded), and at _use_ time you get finally . With the wrong
definition you get at definition time, and this is simply an illegal
character sequence.
PXP defines by default amp; as #38;#38; which is just the same in
decimal notation, and also recommended by the XML spec.
That W3C docs are erroneous is nothing new, although it is a bit
surprising that they cannot even stick to the basics of their own
formalism. I suppose they used a hacked SGML parser for developing
MathML, since SGML is more liberal about lexical details.
Gerd
Though 0x26 is indeed the codepoint for the ampersand character, I don't
get why it appears twice. Is this a case of double escaping? Could this
be the reason PXP chokes?
Any thoughts?
Best regards,
Dario Teixeira
P.S. This is the programme I used for testing. Its code is pretty much
lifted from the PXP manual:
open Pxp_document
open Pxp_yacc
class warner =
object
method warn w = print_endline (WARNING: ^ w)
end
let rec print_structure n =
let ntype = n#node_type
in match ntype with
| T_element name -
print_endline (Element of type ^ name);
let children = n # sub_nodes
in List.iter print_structure children
| T_data -
print_endline Data
| _ -
assert false
let () =
try
let config = {default_config with warner = new warner} in
let doc = parse_document_entity config (from_file test.xml)
default_spec
in print_structure (doc#root)
with
exc - print_endline (Pxp_types.string_of_exn exc)
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
--
Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany
[EMAIL PROTECTED] http://www.gerd-stolpmann.de
Phone: +49-6151-153855 Fax: +49-6151-997714
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] XML library for validating MathML
Am Donnerstag, den 18.09.2008, 13:44 -0700 schrieb Dario Teixeira: Hi, Code points: Note that PXP defaults to ISO-8859-1 as character set. Use it in UTF-8 mode to get rid of these warnings. Ah, thanks, I'll look into that. By the way, is there an API reference for PXP? I noticed you are not using Ocamldoc, and I'm guessing the PXP manual is built from some sort of literate programming tool. Unfortunately, the manual is quite large, and it can be difficult to navigate. I know, it's from pre-ocamldoc times. I'll see what I can do. Gerd That W3C docs are erroneous is nothing new, although it is a bit surprising that they cannot even stick to the basics of their own formalism. I suppose they used a hacked SGML parser for developing MathML, since SGML is more liberal about lexical details. I'm sure floggings will be administered. Anyway, thanks for making the diagnostic on the problem! I am happy to report that once the isonum.ent file is fixed, PXP is able to validate MathML documents against the official DTD. Fixing the isonum.ent was simply a matter of applying Gerd's correction to two lines, 28 and 66. Here is the diff: 28c28 !ENTITY amp #x26;#x00026; !--=ampersand -- --- !ENTITY amp #x26;#x00026; !--=ampersand -- 66c66 !ENTITY lt #x26;#x0003C; !--=less-than sign R: -- --- !ENTITY lt #x26;#x0003C; !--=less-than sign R: -- Cheers, Dario Teixeira -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Findlib fails to build on OS X Leopard
I've released findlib-1.2.3 (hopefully) fixing this problem. Gerd Am Mittwoch, den 16.07.2008, 18:17 -0700 schrieb Nathaniel Gray: My colleague is trying to install GODI but it's choking on findlib 1.2.2. Specifically, the command used to locate the std. lib. in get_stdlib is not compatible with OS X's sed: ocamlc -where | sed s/\r// || ... The version of sed included with Leopard doesn't support backslash-escapes like \r. Instead, it treats '\r' just like 'r', so this turns '/Users/blah/...' into '/Uses/blah/...' and the std. lib. can't be found. Until this bug is fixed the workaround is to install gnu sed. Cheers, -n8 -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: thousands of CPU cores
Am Mittwoch, den 16.07.2008, 10:59 +0200 schrieb Michaël Grünewald: Gerd Stolpmann wrote: Well, there's now SFU for Windows (but only for XP Professional and Windows 2003, not for XP Home and Vista, AFAIK). That's a cool solution when you want to run Win32 and POSIX programs on the same system, and maybe an alternative to using virtualization. But it is nothing for developing consumer programs on Windows. Btw, has something tried to compile O'Caml on SFU? It's a 230M free download. There seems to be gcc and lots of GNU stuff, too (yes, it's from MS...). I did this a few monthes ago, I followed the NetBSD way, since SFU is supported by NetBSD's `pkgsrc'. This was really *easy*, thanks to the efforts of the `pkgsrc' maintainers. However, I did not play that much with the system, my point was to test SFU by running very Unix-oriented and complex proecdures in it. See http://www.netbsd.org/docs/software/packages.html for general information about NetBSD's pkgsrc; Microsoft SFU is refered to as Interix here, e.g. in the ``Supported Platforms'' section. Good to know. The `pkgsrc' software is a port infrastructure similar to what is found on *BSD and MacPorts, if you have used one of them, you certainly will feel comfortable with `pkgsrc'. Documentation for `pkgsrc' is available at http://www.netbsd.org/docs/pkgsrc/, besides the introduction, see especially sections 3.2 (Bootstrapping) and 4.2 (Installing ports), it shall be enough to get started! Yes, I know pkgsrc very well. I used it years ago to build software on Solaris. Later I took it as starting point for GODI. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: thousands of CPU cores
Am Dienstag, den 15.07.2008, 11:57 -0400 schrieb Kuba Ober: On Thursday 10 July 2008, Gerd Stolpmann wrote: Am Donnerstag, den 10.07.2008, 21:02 + schrieb Sylvain Le Gall: On 10-07-2008, Gerd Stolpmann [EMAIL PROTECTED] wrote: Am Donnerstag, den 10.07.2008, 20:07 + schrieb Sylvain Le Gall: On 10-07-2008, Gerd Stolpmann [EMAIL PROTECTED] wrote: In Ocaml you can exploit multi-core currently only by using multi-processing parallel programs that communicate over message passing (and only on Unix). Actually, it's an excellent language for this style. Why only on Unix ? No fork() on Windows. And emulating its effects is hard. open_process + stdin/stdout should do the trick... at least i think so. After having ported godi to mingw I am not sure whether this works at all. The point is that you usually want to inherit OS resources to the child process (e.g. sockets). The CreateProcess Win32 call (http://msdn.microsoft.com/en-us/library/ms682425(VS.85).aspx) mentions that you can inherit handles, but I would be careful with the information given in MSDN. Often it works only as far as the presented examples. Windows isn't written for multi-processing, and its syscalls aren't as orthogonal as in Unix-type systems. Windows syscalls are quite reasonable IMHO, if a tad undocumented. ReactOS folk have done a great job of reimplementing most of them, and there isn't anything mucho broken about those. In fact, I'd posit that Windows native syscalls expose some functionality that's traditionally unavailable on unices and requires hacks to achieve (usually via executable code injection). Just look at what Wine folks had to do in order to emulate some win32 (not even native!) API on Linux: a lot of hard work for what amounts to a single API call. This of course works both ways, and on Windows, while a fork() implementation is simple, it AFAIK requires a custom loader or some other ingenuity to work. Sure, both systems follow different philosophies. Furthermore, it looks like a pain in the ass - often you want to run some initialization code, and without fork() you have to run it as often as you start processes. On Windows, there's the native API, which is then used by the win32 subsystem and posix subsystems to do the job. Native API allows fork() implementations mostly on par with what you get on Unices. MS has a posix subsystem on which fork() performs in the same ballpark as fork() on linux, and make Cygwin's fork() look bad like it deserves. About the only good thing about Cygwin's fork() is that it works on win9x. Well, there's now SFU for Windows (but only for XP Professional and Windows 2003, not for XP Home and Vista, AFAIK). That's a cool solution when you want to run Win32 and POSIX programs on the same system, and maybe an alternative to using virtualization. But it is nothing for developing consumer programs on Windows. Btw, has something tried to compile O'Caml on SFU? It's a 230M free download. There seems to be gcc and lots of GNU stuff, too (yes, it's from MS...). Also, Windows is just a bad platform for event-based programs, and you want to do it to some extent (e.g. for watching all your child processes). Only for socket handles there is a select() call. For all other types of handles you cannot find out in advance whether the operation would block or not. This is misinformation at best, FUD at worst. I'm no Microsoft fanboy, but the reality is quite opposite to what you claim. Windows has quite robust asynchronous I/O support. No, this is not misinformation, this is the result of digging deeply into the Win32 API for an attempt to port Ocamlnet to Win32 (which will finally happen to some degree). There's overlapped I/O, but the difficulty is that you have to start an operation before you can watch asynchronously for its completion. There is no way to check in advance for that (and that was my claim). Also, there is a quite small limit for the number of resources you can watch at the same time (I think 32 or 64). Look at what Cygwin has done. Basically, they start helper threads for emulating select(). For some cases, there is no real select() support, e.g. the output side of pipes is always considered as writable. Only the input side can be watched. Of course, these difficulties result from porting Unix libraries to Windows. You can say with some right: If you want to program in the event-based way on Windows, you must do it the Windows style. Sure, we are running into the same problems as with fork() - different philosophies make it problematic to write portable programs. What's quite interesting is that the Win32 APIs are less powerful in these areas (process creation, watching events) than the Unix counterpart. That's my whole claim. If I had to develop programs only for Windows, I'd do it multi-threaded because Win32 is much better there. Gerd
Re: [Caml-list] thousands of CPU cores
Am Donnerstag, den 10.07.2008, 23:01 -0400 schrieb Brian Hurt: On Thu, 10 Jul 2008, Gerd Stolpmann wrote: I wouldn't take this article too seriously. It's just speculation. I would take the article seriously. Just open up your mind to this perspective: It's a big risk for the CPU vendors to haven taken the direction to multi-core. *Precisely*. It also stands in stark contrast to the last 50 or so years of CPU development, which focused around making single-threaded code faster. And, I note, it's not just one CPU manufacturer who has done this (which could be chalked up to stupid management or stupid engineers)- but *every* CPU manufacturer. And what do they get out of it, other than ticked off software developers grumbling about having to switch to multithreaded code? I can only see one explanation: they had no choice. They couldn't make single threaded code any faster by throwing more transistors at the problem. We've maxed out speculative execution and instruction level parallelism, pushed cache out well past the point of diminishing returns, and added all the execution units that'll ever be used, what more can be done? The only way left to increase speed is multicore. And you still have the steady drum beat of Moore's law- which, by the way, only gaurentees that the number of transistors per square millimeter doubles every yeah so often (I think the current number is 2 years). So we have the new process which gives us twice the number of transistors as the old process, but nothing we can use those new transistors on to make single threaded code go any faster. So you might as well give them two cores where they used to have one. At this point, there are only two things that can prevent kilo-core chips: one, some bright boy could come up with some way to use those extra transistors to make single-threaded code go faster, or two: Moore's law could expire within the next 16 years. We're at quad-core already, another 8 doublings every 2 years, with all doublings spent on more cores, gets us to 1024 cores. Well, it is an open question whether this alternative holds. I mean there is a market, and if the market says, no we don't need that multicore monsters, the chip companies cannot simply ignore it. Of course, there are applications that would extremely benefit from them, but it is the question whether this is only a niche, or something you can make enough revenues to pay the development of such large multicores. In the past, it was very important for hardware vendors that existing software runs quicker on new CPU generations. This is no longer true for multicore. So unless there is a software revolution that makes it simple to exploit multicore, we won't see 1024-cores for the masses. I think it'll be worse than this, actually, once it gets going. The Pentium III (single core) was 9.5 million transistors, while the Core Duo was 291 million. Even accounting for the 2 cores and some cost to put them together, you're looking at the P-III to be 1/16th the size of a Core. If put on the same process so the P-III runs at more or less the same clock speed, how much slower would the P-III be? 1/10th? 1/2? 90% the speed of the Core? So long as it's above 1/16th the speed, you're ahead. If your code can scale that far, it's worthwhile to have 32 P-III cores instead of a the dual core Core Duo- it's faster. Yes, there are limits to this (memory bandwidth springs to mind), but the point is that more, simpler cores could be a performance win, increasing the rate cores multiply faster than Moore's law would dictate. If we decide to go to P-III cores instead of Core cores, we could have 1024-core chips comming out in 8 years (4 doublings, not 8, to go from 4x32=64 cores to 1024 cores). And remember, if Moore's law holds out for another 8 years after we hit 1K cores, that's another 4 doublings, and we're looking at CPUs with 16K cores- literally, tens of thousands of cores. Well, there is a another option for the chip industry. Instead of keeping the die at some size and packing more and more cores on it, they can also sell smaller chips for less. Basically, this alternate path already exists (e.g. Intel's Atom). Of course, this makes this industry more boring, and they would turn into more normal industrial component suppliers. At some point this will be unavoidable. The question is whether this happens in the next years. Gerd If Moore's law doesn't hold up, that's going to be a different, and much larger and smellier, pile of fecal matter to hit the rotary air impeller. Brian ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] GODI News: RocketBoost Beta
Am Sonntag, den 15.06.2008, 23:42 +0200 schrieb Adrien: Can you check whether the newly installed godi_console works? A good test is godi_console perform -build godi-tools so it builds itself again using the new godi_console (and not the bootstrap version). Well, it doesn't work : [EMAIL PROTECTED] ~ $ godi_console perform -build godi_tools Fatal error: cannot find file perform I think I got it. It may happen that there are both godi_console (as shell script) and godi_console.exe, and depending on the Cygwin version one or the other is called, and for some versions invocation fails completely. Just delete the shell scripts (in the bin and sbin directories, when there is also an .exe), and it should work. I've released a new version of rocketboost that fixes this: http://download.camlcity.org/download/godi-rocketboost-20080616.tar.gz Gerd I had some weird feeling about this so I tried strings on godi_console and in fact... [EMAIL PROTECTED] ~ $ cat `which godi_console` #! /bin/sh exec /home/Administrateur/godi-rocketboost-20080615/run/ocamlrun-3.09.3/byterun/ocamlfatrun /home/Administrateur/godi-rocketboost-20080615 /godi-tools-boot/boot_console $@ I guess it failed at some point during installation but I really don't know when. I'll try another installation on tomorrow. I have time to install godi from scratch so if you need anything, don't hesitate. :) Thanks for your help. Btw, is there any chance the PATH gets automatically updated ? I've installed ocaml by hand several times but I keep on forgetting this step. Well, you would need to do it in your login setup, e.g. .bashrc. The bootstrap script just cannot do it. What we can talk about is that stage2 is automatically called. I think it's alright this way. I just always forget the PATH thing. I think I'll have a tatoo or something like that, that should do the trick. --- Adrien Nader 2008/6/15 Gerd Stolpmann [EMAIL PROTECTED]: Hi, In the past weeks I've worked hard to finish the next GODI release, focusing on portability. A beta version of the new bootstrap and godi_console, called RocketBoost, is now available, and it would be great if it were tested at large. There are not many new features in this release, so there is no reason to switch to it if you already have GODI. The big news is that GODI now supports the MinGW port of OCaml for Windows (besides the Cygwin port). This means that it is now possible to create native Windows applications with GODI. Note, however, that the porting effort is still in its beginning - GODI itself works, but most packages aren't ported yet, and won't work. The MinGW support has become possible because large parts of the GODI core have been rewritten. In particular, the C programs accompanying godi_console are now coded in O'Caml (godi_make, godi_add, etc.), so porting was possible by enhancing a few O'Caml libraries. This refactoring of GODI also increases the portability in the Unix world - effectively it should now run on all platforms where OCaml runs, where gcc is available, and where POSIX-compliant shell utilities are available. If you would like to test it, follow these instructions. For all ports, you need the bootstrap tarball from: http://download.camlcity.org/download/godi-rocketboost-20080615.tar.gz --- Installation for Unix --- For Unix platforms (including MacOSX), just download godi-rocketboost-20080615.tar.gz, unpack it, run ./bootstrap, and follow the instructions. --- Installation for Windows --- For Windows you always need Cygwin, even for the MinGW port (it is needed as scripting environment, please don't question that). Get it from cygwin.com. Install everything that is needed (binutils, bzip2, diffutils, file, findutils, gawk, gcc-core, gcc-mingw-core, grep, groff, gzip, m4, make, man, mingw-runtime, patch, rxvt, sed, tar, w32api, wget - hope this list is complete). IMPORTANT: When you install software you need for GODI, choose paths that do not contain space characters. This is incompatible with the shell scripts. So don't install into My Files. Download godi-rocketboost-20080615.tar.gz, and unpack it: $ tar xzf godi-rocketboost-20080615.tar.gz $ cd godi-rocketboost-20080615 Now invoke bootstrap. You probably want to set the path where it is going to be installed with -prefix. Furthermore, you can now select whether you want the Cygwin or the MinGW port. For the latter, pass -w32port mingw as extra argument. IMPORTANT: Remember that GODI itself relies on Cygwin scripting. If you pass paths to GODI (including godi_console, godi_add, etc.) these are Cygwin paths, even if you build the MinGW port. $ ./bootstrap -prefix /home/gerd/godi -w32port mingw This will build boot_console. Then you
Re: [Caml-list] How to compile using sexplib?
Am Donnerstag, den 12.06.2008, 09:44 -0700 schrieb Luca de Alfaro: I am trying to use sexplib to serialize some data structures, as many of you advised, and I am somewhat stuck trying to compile. This is what I get: ocamlfind ocamlc -package unix,str,vec,mapmin,hashtbl_bounded,fileinfo,intvmap,extlib,mysql,sexplib -I ../../batch/analysis -I +camlp4 -pp camlp4orf -g -c online_db.ml File online_db.ml, line 44, characters 22-26: Parse error: [semi] expected after [str_item] (in [implem]) Preprocessor error I would try ocamlfind ocamlc -package sexplib,... -syntax camlp4o -I ... -c online_db.ml ocamlfind includes some mechanics to drive camlp4, and I am pretty sure sexplib configures that. Gerd where my line 44 is simply: type foo_t = int list with sexp I have the impression that the problem is in how I compile. Specifically, if I do not use the -pp camlp4orf part, I get a syntax error, as the file is not getting preprocessed. If I use the above -pp line, I get the error I reported. I am not sure what is the proper way to preprocess the file, and unfortunately, sexplib does not explain how to compile code against it. I am sure this is obvious for everybody, but it is the first time I use camlp4, so some help would be appreciated. All the best, Luca ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] HTTP Client for OCaml
Am Montag, den 02.06.2008, 15:14 -0500 schrieb Robert Fischer: Is there an HTTP client which can 1) do HTTP authentication and 2) report on status codes? I've tried both libcurl and ocamlnet, and encountered serious problems in both -- libcurl reports that CURLINFO_RESPONSE_CODE isn't implemented, and ocamlnet throws steaming assertion death if authentication fails. So I'm looking for a new option. For example, fixing the errors? I could do that for ocamlnet, given I have a description of the error case. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Where's my non-classical shared memory concurrency technology?
Am Mittwoch, den 21.05.2008, 09:06 +0100 schrieb Martin Berger: Here I disagree. Shared memory concurrency is a specific form of message passing: Writing to a memory cell is in fact sending a message to that cell carrying two items, the new value and a return channel that is used to inform the writer that sending has succeeded, and likewise for reading. This way of thinking about shared memory concurrency has enabled the construction of powerful logics and typing systems to reason about shared memory. But this is still academic, right? The practice is that there is almost no utility that helps programmers to write (more) correct code. I agree with the spirit of your claim though: shared memory is a form of message passing that is especially difficult. With reasoning I don't necessarily mean formal techniques. The more frequent case is that the programmer thinks about the program guided by the laws of logic. The impossibility to do this with truly parallelized code is an important source of bugs, so I would say this code inherently more buggy. There is a lot of truth in what you say, but if you consider complex event handling programs, you essentially do concurrent programming, wether you like it or not. You just don't call it by that name. Of course, it is possible to e.g. develop locking mechanism in event handling programs, and then to run into the same problems. My experience is different, however. The worst thing I've encountered so far in practice is the forgotten continuation so that the event system freezes. Another result from using this is in practice is that it is comparatively easy to debug synchronization problems, because the debugging code is not itself concurrent code. This is different to e.g. multi-threaded programs where the whole program is concurrent, and you cannot locally escape from this environment. This is simply nonsense. Different concurrency techniques have different problems. For example, in event handling-based concurrency you do not need locks, hence you cannot run into deadlocks. I disagree, but as the issue has already been dealt with by other posters, I shall leave it at that. Granted, cannot run into deadlocks is too strong. Maybe in a mainstream language, but in FP I found it always relatively easy to do it. Maybe you are an especially skilled programmer. Or maybe the applications that you have coded are not demanding in terms of concurrency. Well, it's probably one of the biggest programs that has ever been coded in O'Caml... It's a search engine (wink.com), and I would say it is very demanding in terms of concurrency. Keep in mind that I am not a researcher. When I make statements, then from a practical point of view. My observation is simply that you run far quicker into complicated synchronization problems with shared memory concurrency than with the types of concurrency I prefer. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Why OCaml sucks
Am Dienstag, den 13.05.2008, 10:01 -0400 schrieb Brian Hurt: Robert Fischer wrote: 5. Strings: pushing unicode throughout a general purpose language is a mistake, IMHO. This is why languages like Java and C# are so slow. Unicode by itself, when wider-than-byte encodings are used, adds zero runtime overhead; the only overhead is storage (2 or 4 bytes per character). You cannot degrade memory consumption without also degrading performance. Moreover, there are hidden costs such as the added complexity in a lexer which potentially has 256x larger dispatch tables or an extra indirection for every byte read. Okay, I was going to let this slide, but it kept resurfacing and annoying me. Is there any empirical support for the assertion that Java and C# are slow because of *unicode*? Of all things, *unicode*? The fact that they're bytecod languages isn't a bigger hit? At least with the JVM, the hypercomplicated GC should probably take some of the blame, too -- I've seen 2x speed increases by *reducing* the space available to the GC, and 10x speed increases by boosting the space available to ridiculous levels so that the full GC barely ever has to fire. The the nigh-universal optimization-ruining mutable data and virtual function (e.g. method) dispatch I'm sure doesn't help, too. And this is to say nothing of user-space problems like the explosion of nontrivial types associated with the object-driven style. With all that going on, you're blaming their *Unicode support* for why they're slow? This is why languages like Java and C# are so slow. Really? Got evidence for that? ~~ Robert. _ The problem, as I understand it, is in writting parsers. Your standard finite automata based regular expression library or lexical analyzer is based, at it's heart, on a table lookup- you have a 2D array, whose size is the number of input characters times the number of states. For ASCII input, the number of possible input characters is small- 256 at most. 256 input characters times hundreds of states isn't that big of a table- we're looking at sizes in 10's of K- easily handlable even in the bad old days of 64K segments. Even going to UTF-16 ups the number of input characters from 256 to 65,536- and now a moderately large state machine (hundreds of states) weighs in at tens of megabytes of table space. And, of course, if you try to handle the entire 31-bit full unicode point space, welcome to really large tables :-). The solution, I think, is to change the implementation of your finite automata to use some data structure smarter than a flat 2D array, but that's me. Note that we have such a lexer already: ulex. It uses binary decision trees, AFAIK. The resulting code has moderate size. It can take some time, however, until ulex has converted the NFA to a DFA. Gerd -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED] http://www.gerd-stolpmann.de Phone: +49-6151-153855 Fax: +49-6151-997714 ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Re: Why OCaml rocks
Am Freitag, den 09.05.2008, 06:09 +0100 schrieb Jon Harrop: On Friday 09 May 2008 05:45:53 you wrote: On Thu, May 8, 2008 at 5:39 PM, Jon Harrop [EMAIL PROTECTED] wrote: 1. Lack of Parallelism: Yes, this is already a complete show stopper. Exploiting multicores requires a scalable concurrent GC and message passing (like JoCaml) is not a substitute. Unfortunately, this is now true of all functional languages available for Linux, which is why we have now migrated entirely to Windows and F#. I find it particularly ironic that the Haskell community keep hyping the multicore capabilities of pure code when the rudimentary GC in Haskell's only usable implementation already stopped scaling. Fork? For something like a raytracer, I do not see how threads would be any more useful than fork. I think the parallelism capabilities are already excellent. We have been able to implement the application backend of Wink's people search in O'Caml, and it is of course a highly parallel system of programs. This is not the same class raytracers or desktop parallelism fall into - this is highly professional supercomputing. I'm talking about a cluster of ~20 computers with something like 60 CPUs. Of course, we did not use multithreading very much. We are relying on multi-processing (both forked style and separately started programs), and multiplexing (i.e. application-driven micro-threading). I especially like the latter: Doing multiplexing in O'Caml is fun, and a substitute for most applications of multithreading. For example, you want to query multiple remote servers in parallel: Very easy with multiplexing, whereas the multithreaded counterpart would quickly run into scalability problems (threads are heavy-weight, and need a lot of resources). There are two problems with that: . You go back to manual memory management between parallel threads/processes. I guess you refer to explicit references between processes. This is a kind of problem, and best handled by avoiding it. We have some cases where we have to keep remote state. The solution was to have a timer, and delete it after some time of not accessing it. After all, most state is only temporary, and if it is lost, it can be created again (at some cost, of course). . Parallelism is for performance and performance requires mutable data structures. In our case, the mutable data structures that count are on disk. Everything else is only temporary state. I admit that it is a challenge to structure programs in a way such that parallel programs not sharing memory profit from mutable state. Note that it is also a challenge to debug locks in a multithreaded program so that they run 24/7. Parallelism is not easy after all. Then you almost always end up copying data unnecessarily because you cannot collect it otherwise, which increases memory consumption and massively degrades performance that, in turn, completely undermines the original point of parallelism. Ok, I understand. We are complete fools. :-) I think that the cost of copying data is totally overrated. We are doing this often, and even over the network, and hey, we are breaking every speed limit. The cost of interthread communication is then so high in OCaml that you will rarely be able to obtain any performance improvement for the number of cores desktop machines are going to see over the next ten years, by which time OCaml will be 10-100x slower than the competition. This is a quite theoretical statement. We will rather see that most application programmers will not learn parallelism at all, and that consumers will start question the sense of multicores, and the chip industry will search for alternatives. And _if_ application programmers learn parallelism, then rather in the multi-processing/multiplexing setup we use, and not the multithreading style you propagate. And on servers (where parallelism ought to happen), the poor support of Windows for it (lacking fork and other cool features) is no problem. Gerd When was the last time you heard of a cool new windows app anyway? The last time we released a product. :-) . No 16Mb limit. What do you mean by 16mb limit? OCaml's strings and arrays are limited to 16Mb in 32-bit. . Inlining. isn't it best for the compiler to handle that? I wouldn't mind hearing another perspective on this, but I thought that compilers were smarter these days. Definitely not. Compilers uniformly suck at inlining. For example, agressive inlining is often beneficial in numerical code and often damaging in symbolic code. Compilers cannot tell the difference. This is very similar to unboxed data structures are always better, which also isn't generally true. I've got more gripes to add: . Missing types, like float32 and int16. . DLLs. -- Gerd Stolpmann * Viktoriastr. 45 * 64293 Darmstadt * Germany [EMAIL PROTECTED
