On Tuesday, 28 October 2014 at 09:34:33 UTC, Chris wrote:
On Tuesday, 28 October 2014 at 00:16:03 UTC, John McFarlane
wrote:
Hi,
I've written a modest shared library in D that I'd like to
call directly from a Python web server (Linux/OS X, Apache,
WSGI, Pyramid). I can call it reliably from within Python unit
tests but on a running server, the library causes a SIGSEGV as
soon as I try anything as complicated as a writeln call. Under
Linux, I've tried using Pyd, CFFI and ctypes to bind the
native .so to Python with the same result. I've tried calling
Runtime.initialize();from an init function on server startup
and from the D function being called as part of the web
request. I've even tried writing a shim in C to make these
calls and call through to the D.
I've compiled with DMD and GDC with a variety of switches.
Currently my build command looks like:
dmd ./d/*.d -version=library -release -shared -O -inline
-defaultlib=libphobos2.so -fPIC -oflibgusteau.so
The extern(C) function takes a string and returns a string.
I've tried holding onto a pointer to the returned string and
passing in an adequately sized byte array from Python. Smaller
strings seem to cause a crash with lower probability but
typically the input and output strings are a few KB in size
and consistently crash. I taken measures to ensure that they
are correctly encoded and null terminated. I've also tried
disabling the GC, calling terminate on function exit (after
copying result into received buffer) and various other
measures in an attempt to get something working.
I'm guessing that the web server spawns/relies on a thread or
process for each request and I've read that runtime
initialization should be invoked from the main thread. Does
this always mean the first thread that was created on process
start up or does it just have to be consistent? If calling
from a thread other than the one used to initialize, is that a
problem? Are other threads created when GC is invoked which
might last past the extern function being called and if so,
can I prevent this by controlling collection explicitly?
Thanks, John
I had a similar problem with a DLL for Python. The reason my
DLL would crash were some writeln statements. After removing
them it worked fine. I suppose the writeln messes things up, be
it because of threads or because of conflicts in the file
system (remember writeln is a file system operation). I suggest
you have your lib not print things to console, but use a
different mechanism instead, e.g. pass a string back to Python
and have Python do the printing (if needs be).
You could also try to set your lib up as a small socket server
that waits for input and answers. That usually works for me.
Thanks for the suggestion. I looked into removing all of my
writeln calls and
found the I still had the same problem. Simply a call to
to!string with the
input char* is enough to cause a crash. I will go with the socket
server as a
last resort but it would be much easier (especially from a
testing POV) if I
could call the native D code directly from the web server.
