[Caml-list] Question about ocaml threads and TLS (on linux)
Hi,
I'm having a little problem for my libfuse-ocaml bindings for the
threaded interface. For those that don't want to read all of the mail my
question is:
Will every ocaml thread have its own thread-local-storage in the C stubs?
I have the following calling sequence:
User ocaml code | Fuse C stub | libfuse code
--+--+-
Fuse.process fs'process stub' |
| enter_blocking_section() |
| char *buf = malloc(size) |
| fuse_session_process()
| ops-write(buf+off)
| 'write stub' |
| leave_blocking_section() |
| a = caml_ba_alloc_dims() |
caml_callback(...,a,...) |
my_ref := a |
| enter_blocking_section()
| callback done
| 'process stub' |
| free(buf)|
leave_blocking_section() |
Fuse.process done |
The 'process stub' allocates a buffer and frees it at the end, which is
usualy fine. Except in the case of a write callback where the buffer is
passed back to ocaml as Bigarray. If the Bigarray is copied, like above,
then the ocaml code still has a reference to the data at the point the
'process stub' wants to free it.
To solve that problem I need the write callback to signal that the
buffer was passed to ocaml and is now under GC control. The buffer must
not be free()ed by the 'process stub'. The libfuse API does not provide
for this so I have to somehow communicate between 'process stub' and
'write stub' around the libfuse code.
Possible solution:
--
__thread char *buf = NULL;
value ocaml_fuse_process(...) {
buf = malloc(size);
fuse_session_process()
if (buf != NULL) free(buf);
}
void write_callback(...) {
a = caml_ba_alloc_dims(...);
buf = NULL;
}
This way ocaml_fuse_process will allocate a new buffer whenever it
doesn't have one and the write_callback will take over the buffer and
give it to the GC.
Now my question is: Does that work? Is it safe? Will every ocaml thread
have its own thread-local-storage buf?
Currently I'm only interested in supporting Linux. If it is safe there
that is enough.
MfG
Goswin
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Question about ocaml threads and TLS (on linux)
Hi,
I'm not sure I understand (though I've read the whole text), but maybe
this will answer your question:
On Linux, OCaml threads (with the native compiler ocamlopt) are
implemented with POSIX threads (in C), so when your OCaml thread runs
the C stub, it's the same as if you were running the C stub in some C
thread.
When you are in a section declared as a blocking section, a collection
can be triggered concurrently in another thread and so the heap must
not be accessed neither for reading or writing, that's all.
Using __thread recent feature should also work if you manage to
compile everything correctly. Notably, we use it in some places in
ocaml4multicore (a patch to ocaml's runtime library to allow parallel
threads). However, I don't know how __thread is handled by the
compiler... I mean : it there a pointer for buf in every thread or
only in those that use it?
I hope my answer isn't useless!
Cheers,
--
Philippe Wang
m...@philippewang.info
On Wed, Feb 24, 2010 at 10:00 PM, Goswin von Brederlow
goswin-...@web.de wrote:
Hi,
I'm having a little problem for my libfuse-ocaml bindings for the
threaded interface. For those that don't want to read all of the mail my
question is:
Will every ocaml thread have its own thread-local-storage in the C stubs?
I have the following calling sequence:
User ocaml code | Fuse C stub | libfuse code
--+--+-
Fuse.process fs 'process stub' |
| enter_blocking_section() |
| char *buf = malloc(size) |
| fuse_session_process()
| ops-write(buf+off)
| 'write stub' |
| leave_blocking_section() |
| a = caml_ba_alloc_dims() |
caml_callback(...,a,...) |
my_ref := a |
| enter_blocking_section()
| callback done
| 'process stub' |
| free(buf) |
leave_blocking_section() |
Fuse.process done |
The 'process stub' allocates a buffer and frees it at the end, which is
usualy fine. Except in the case of a write callback where the buffer is
passed back to ocaml as Bigarray. If the Bigarray is copied, like above,
then the ocaml code still has a reference to the data at the point the
'process stub' wants to free it.
To solve that problem I need the write callback to signal that the
buffer was passed to ocaml and is now under GC control. The buffer must
not be free()ed by the 'process stub'. The libfuse API does not provide
for this so I have to somehow communicate between 'process stub' and
'write stub' around the libfuse code.
Possible solution:
--
__thread char *buf = NULL;
value ocaml_fuse_process(...) {
buf = malloc(size);
fuse_session_process()
if (buf != NULL) free(buf);
}
void write_callback(...) {
a = caml_ba_alloc_dims(...);
buf = NULL;
}
This way ocaml_fuse_process will allocate a new buffer whenever it
doesn't have one and the write_callback will take over the buffer and
give it to the GC.
Now my question is: Does that work? Is it safe? Will every ocaml thread
have its own thread-local-storage buf?
Currently I'm only interested in supporting Linux. If it is safe there
that is enough.
MfG
Goswin
___
Caml-list mailing list. Subscription management:
http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list
Archives: http://caml.inria.fr
Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
Bug reports: http://caml.inria.fr/bin/caml-bugs
Re: [Caml-list] Question about ocaml threads and TLS (on linux)
Philippe Wang philippe.wang.li...@gmail.com writes: Hi, I'm not sure I understand (though I've read the whole text), but maybe this will answer your question: On Linux, OCaml threads (with the native compiler ocamlopt) are implemented with POSIX threads (in C), so when your OCaml thread runs the C stub, it's the same as if you were running the C stub in some C thread. How about bytecode? When you are in a section declared as a blocking section, a collection can be triggered concurrently in another thread and so the heap must not be accessed neither for reading or writing, that's all. Using __thread recent feature should also work if you manage to compile everything correctly. Notably, we use it in some places in ocaml4multicore (a patch to ocaml's runtime library to allow parallel threads). However, I don't know how __thread is handled by the compiler... I mean : it there a pointer for buf in every thread or only in those that use it? Only in those that use it. But that will be multiple threads. In those that don't use it 4/8 bytes will be wasted. I hope my answer isn't useless! Cheers, MfG Goswin ___ Caml-list mailing list. Subscription management: http://yquem.inria.fr/cgi-bin/mailman/listinfo/caml-list Archives: http://caml.inria.fr Beginner's list: http://groups.yahoo.com/group/ocaml_beginners Bug reports: http://caml.inria.fr/bin/caml-bugs
