Re: Python calling D
On Tuesday, 4 February 2014 at 16:17:30 UTC, Russel Winder wrote: On Tue, 2014-02-04 at 12:45 +, Artem Tarasov wrote: But it does lead to a working system :-) Any particular reason you aren't using CeleriD to build this shared lib? CeleriD uses some hooks to call rt_init when the library loads. https://bitbucket.org/ariovistus/pyd/src/92b9962b429ed33afa7048cf1923fd76d0fe8977/infrastructure/d/?at=default see so_ctor.c and python_so_linux_boilerplate.d Also, note however you do it, if you have multiple shared libs, don't call initialize/term more than once. segfaults happen.
Re: Python calling D
On Tue, 2014-02-04 at 07:13 +, Artem Tarasov wrote:
On Sunday, 2 February 2014 at 15:31:30 UTC, Russel Winder wrote:
result is:
| LD_LIBRARY_PATH=. python execute.py
Segmentation fault
You should call Runtime.initialize() prior to calling any other D
functions.
Hummm… obvious once pointed out :-) Thanks for doing exactly that.
The question is how to get this run. I tried a module initializer:
static this() {
Runtime.initialize();
}
but this module never actually gets loaded per se, so that is never
going to work. So this means calling a C linkage function to do the
initialization in every entry point:
import core.runtime: Runtime;
import std.stdio: writeln;
auto initialized = false;
extern (C) void initializeIfNotAlreadyDone() {
if (!initialized) {
Runtime.initialize();
initialized = true;
}
}
extern(C) {
void sayHello() {
initializeIfNotAlreadyDone();
writeln(Hello World.);
}
}
works a treat:
scons: Building targets ...
dmd -I. -fPIC -c -fPIC -ofhelloWorld.o helloWorld.d
dmd -shared -defaultlib=libphobos2.so -oflib_helloWorld.so helloWorld.o
-fPIC -L-L.
LD_LIBRARY_PATH=. python execute.py
Hello World.
scons: done building targets.
So D extensions to CPython are possible without PyD!
Now to continue with PyD anyway so as to compare and contrast.
--
Russel.
=
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
On Tuesday, 4 February 2014 at 11:33:40 UTC, Russel Winder wrote:
The question is how to get this run.
Pointing out obvious things, part 2: wrap it into a C function
and call that function when loading the Python module.
library.d:
...
extern (C) export void attach() { Runtime.initialize(); }
library/__init__.py:
...
lib = ctypes.CDLL(myawesomelib.so)
lib.attach()
def foo():
lib.foo()
...
Re: Python calling D
On Tue, 2014-02-04 at 12:45 +, Artem Tarasov wrote:
On Tuesday, 4 February 2014 at 11:33:40 UTC, Russel Winder wrote:
The question is how to get this run.
Pointing out obvious things, part 2: wrap it into a C function
and call that function when loading the Python module.
I had thought of this and rejected it as an API fail. (Possibly wrongly,
but…)
library.d:
...
extern (C) export void attach() { Runtime.initialize(); }
library/__init__.py:
...
lib = ctypes.CDLL(myawesomelib.so)
lib.attach()
This violates the principle of least surprise: you don't have to do this
with C or C++ extensions so it is an API fail for D to have to do this.
def foo():
lib.foo()
...
But it does lead to a working system :-)
--
Russel.
=
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
On Sunday, 2 February 2014 at 15:31:30 UTC, Russel Winder wrote: result is: | LD_LIBRARY_PATH=. python execute.py Segmentation fault You should call Runtime.initialize() prior to calling any other D functions.
Re: Python calling D
On Sunday, 26 January 2014 at 14:17:18 UTC, Russel Winder wrote: On Sun, 2014-01-26 at 12:11 +, Russel Winder wrote: […] However with Python 2 the example from: https://bitbucket.org/ariovistus/pyd/wiki/QuickStart leads to: This all sounds suspiciously like stuff I thought I'd already fixed. see https://bitbucket.org/ariovistus/pyd/issue/15/some-issues-getting-pyd-to-work
Re: Python calling D
On Sat, 2014-02-01 at 20:58 +, Ellery Newcomer wrote: On Sunday, 26 January 2014 at 14:17:18 UTC, Russel Winder wrote: On Sun, 2014-01-26 at 12:11 +, Russel Winder wrote: […] However with Python 2 the example from: https://bitbucket.org/ariovistus/pyd/wiki/QuickStart leads to: This all sounds suspiciously like stuff I thought I'd already fixed. see https://bitbucket.org/ariovistus/pyd/issue/15/some-issues-getting-pyd-to-work The problem was the source → sources edit: Python 2 does not give a reasonable error message, Python 3 does and leads to an immediate fix :-) My problem of the moment is segmentation faults during execution, and I have no model of how to go about providing useful data to debug this :-(( -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
On Saturday, 1 February 2014 at 22:02:24 UTC, Russel Winder wrote: My problem of the moment is segmentation faults during execution, and I have no model of how to go about providing useful data to debug this :-(( It wouldn't by any chance be related to https://bitbucket.org/ariovistus/pyd/issue/17/it-seems-py_incref-missed-when-pydobject would it?
Re: Python calling D
On Sun, 2014-01-26 at 01:33 +, Ellery Newcomer wrote: On Friday, 24 January 2014 at 10:55:34 UTC, Russel Winder wrote: Probably want to use a virtualenv for this rather than install into the base installation you can also do python setup.py build python runtests.py -b hello It needs to work for Python 3.3 as well! try the latest commit OK, that works for me, at least as far as installing is concerned. You probably want to reject my pull request given the boundary conditions you want to work with. However with Python 2 the example from: https://bitbucket.org/ariovistus/pyd/wiki/QuickStart leads to: | python setup.py build Traceback (most recent call last): File setup.py, line 11, in module d_lump=True File /home/users/russel/PythonEnvironments/Fedora_Python2_PyD/lib/python2.7/site-packages/celerid/support.py, line 53, in __init__ std_Extension.__init__(self, *args, **kwargs) TypeError: __init__() takes at least 3 arguments (2 given) on Fedora 20 using a virtualenv created from the standard Python 2. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
On Sun, 2014-01-26 at 12:11 +, Russel Winder wrote: […] However with Python 2 the example from: https://bitbucket.org/ariovistus/pyd/wiki/QuickStart leads to: | python setup.py build Traceback (most recent call last): File setup.py, line 11, in module d_lump=True File /home/users/russel/PythonEnvironments/Fedora_Python2_PyD/lib/python2.7/site-packages/celerid/support.py, line 53, in __init__ std_Extension.__init__(self, *args, **kwargs) TypeError: __init__() takes at least 3 arguments (2 given) on Fedora 20 using a virtualenv created from the standard Python 2. Of course using the far superior Python 3: | python setup.py build Traceback (most recent call last): File setup.py, line 13, in module d_lump=True File /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/support.py, line 53, in __init__ std_Extension.__init__(self, *args, **kwargs) TypeError: __init__() missing 1 required positional argument: 'sources' so srcs → sources cures the problem. I get a dmd error thought: dmd -property -c -fPIC -version=PydPythonExtension -version=Python_2_4_Or_Later -version=Python_2_5_Or_Later -version=Python_2_6_Or_Later -version=Python_2_7_Or_Later -version=Python_3_0_Or_Later -version=Python_3_1_Or_Later -version=Python_3_2_Or_Later -version=Python_3_3_Or_Later -version=Python_Unicode_UCS4 -debug -I/home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure -ofbuild/temp.linux-x86_64-3.3/infra/temp.o hello.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/class_wrap.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/ctor_wrap.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/def.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/embedded.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/exception.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/extra.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/! infrastructure/pyd/func_wrap.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/make_object.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/make_wrapper.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/op_wrap.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/pyd.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/pydobject.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/references.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/pyd/struct_wrap.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/cele! rid/infrastructure/util/conv.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/util/typeinfo.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/util/typelist.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/util/multi_index.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/util/replace.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/meta/Demangle.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/meta/Nameof.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/deimos/python/abstract_.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/deimos/python/ast.d /home/users/russel/Pyt! honEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/deimos/python/boolobject.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/deimos/python/bufferobject.d /home/users/russel/PythonEnvironments/Fedora_Python3_PyD/lib/python3.3/site-packages/celerid/infrastructure/deimos/python/bytearrayobject.d
Re: Python calling D
On Friday, 24 January 2014 at 10:55:34 UTC, Russel Winder wrote: Probably want to use a virtualenv for this rather than install into the base installation you can also do python setup.py build python runtests.py -b hello It needs to work for Python 3.3 as well! try the latest commit
Re: Python calling D
On Fri, 2014-01-24 at 02:29 +, Ellery Newcomer wrote: […] I have just tried a trivial D source shared object on Debian Unstable using DMD 2.064.2 from d-apt. Compile up the shared object with entries C linkage, try to use ctypes or CFFI from Python just gives a segmentation violation :-( python wants shared libs, not static libs. but this is very fiddly. so use pyd. 1. clone the ariovistus/pyd repo 2. from top dir, type python setup install python runtests.py hello Probably want to use a virtualenv for this rather than install into the base installation (or cd examples/hello python setup.py build python test.py) this will run the code found in examples/hello (it's a simple python calls D) use python 2.7 and have dmd 2.064 on your path somewhere. It should just work. It needs to work for Python 3.3 as well! -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
On Wed, 2013-12-11 at 12:56 +0100, John Colvin wrote: On Wednesday, 11 December 2013 at 11:41:11 UTC, Chris wrote: […] Have you had a look at this: http://pyd.dsource.org/ https://github.com/dansanduleac/pyd both of those are out of date, this is where development is now: https://bitbucket.org/ariovistus/pyd Sorry to be late coming to this. It would great to be able to push D as a CPython extension language. However the state of pyd.dsource.org and places reached from it do make it seem that the project died in 2009. ariovistus' GitHub project on Bitbucket is moving but everything else appears to be a problem. Is it possible to decide on a location for activity, create some infrastructure, and then get the old stuff cleaned out. My start point is to get PyD working in virtualenvs for Python 2.7 and Python 3.3. Findout out what the SCons and Tup stuff is for would be useful. On the side I could help with documentation and stuff, but only if it can be done by DVCS and pull requests with email, I am not interested in non DVCS wikis nor forums. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: Python calling D
Sorry to be late coming to this. It would great to be able to push D as a CPython extension language. However the state of pyd.dsource.org and places reached from it do make it seem that the project died in 2009. ariovistus' GitHub project on Bitbucket is moving but everything else appears to be a problem. Is it possible to decide on a location for activity, create some infrastructure, and then get the old stuff cleaned out. My start point is to get PyD working in virtualenvs for Python 2.7 and Python 3.3. Findout out what the SCons and Tup stuff is for would be useful. On the side I could help with documentation and stuff, but only if it can be done by DVCS and pull requests with email, I am not interested in non DVCS wikis nor forums. For future reference, I tried but wasn't able to make calling D from python work. I tried to just compile D to an .a library, including statically linking phobos, and then wrapping it with cython like I would with C code. (I've made this work before with C.) My thinking was that there's already been a lot of work in the Python world in the past 4-6 years to make calling C fairly straight forward. So D should be able to piggy back on that work. (Though to a lot of Python people a legitimate question would be, is this worth it? You can just profile python code to find the hot spots and then use ctypes/cython to pass objects allocated by Python to C-level functions to do the time-intensive stuff.)
Re: Python calling D
On Wednesday, 11 December 2013 at 05:30:49 UTC, CJS wrote: I'd like to use cython to wrap a D library. It's possible to do this with a statically compiled C library, but it fails when I try with a statically compiled D library. Any suggestions on how to do this successfully? I'm not Cython guy, but as I know, Cython generates C code, right? Then probably at first you should learn how to call D functions from C side. I don't know how to do it correctly. It's easily to call those D functions which don't use 'new' operation and garbage collection (explicitly or implicitly). But when they do that, I get segmentation fault.
Re: Python calling D
On Wednesday, 11 December 2013 at 05:30:49 UTC, CJS wrote: I'd like to use cython to wrap a D library. It's possible to do this with a statically compiled C library, but it fails when I try with a statically compiled D library. Any suggestions on how to do this successfully? Have you had a look at this: http://pyd.dsource.org/ https://github.com/dansanduleac/pyd
Re: Python calling D
On Wednesday, 11 December 2013 at 11:41:11 UTC, Chris wrote: On Wednesday, 11 December 2013 at 05:30:49 UTC, CJS wrote: I'd like to use cython to wrap a D library. It's possible to do this with a statically compiled C library, but it fails when I try with a statically compiled D library. Any suggestions on how to do this successfully? Have you had a look at this: http://pyd.dsource.org/ https://github.com/dansanduleac/pyd both of those are out of date, this is where development is now: https://bitbucket.org/ariovistus/pyd
Re: Python calling D
On Wednesday, 11 December 2013 at 05:30:49 UTC, CJS wrote: I'd like to use cython to wrap a D library. It's possible to do this with a statically compiled C library, but it fails when I try with a statically compiled D library. Any suggestions on how to do this successfully? 1) have you declared the functions you want to call with extern(C)? 2) you will need to compile the D code as a shared library. 3) you will need to call rt_init / rt_term to start / stop the runtime in order to use some features of D. There are problems with the runtime when using multiple D shared libraries, so it's best to link everything you need as a single shared lib.
