I have a few radical suggestions:
1. Use ctypes as glue to the core DLL, so we can completely forget about
refcounts and similar mess. Why put manual reference counting and error
handling in the core? It's stupid.
2. The core should be a plain DLL, loadable with ctypes. (I know David
Cournapeau and Robert Kern is going to hate this.) But if Python can
have a custom loader for .pyd files, so can NumPy for it's core DLL. For
ctypes we just need to specify a fully qualified path to the DLL, which
can be read from a config file or whatever.
3. ctypes will take care of all the mess regarding the GIL. Again there
is no need to re-invent the wheel. ctypes.CDLL releases the GIL when
calling into C, and re-acquires before making callbacks to Python.
ctypes.PyDLL keeps the GIL for legacy library code that are not threadsafe.
ctypes will also make porting to other Python implementations easier (or
even other languages: Ruby, JacaScript) easier. Not to mention that it
will make NumPy impervious to changes in the Python C API.
4. Write the core in C++ or Fortran (95/2003), not C. ANSI C (aka C89).
Use std::vector<> instead of malloc/free for C++, and possibly templates
for generics on various dtypes.
5. Allow OpenMP pragmas in the core. If arrays are above a certain size,
it should switch to multi-threading.
Sturla
Den 10.06.2010 15:26, skrev Charles R Harris:
A few thoughts came to mind while reading the initial writeup.
1) How is the GIL handled in the callbacks.
2) What about error handling? That is tricky to get right, especially
in C and with reference counting.
3) Is there a general policy as to how the reference counting should
be handled in specific functions? That is, who does the reference
incrementing/decrementing?
4) Boost has some reference counted pointers, have you looked at them?
C++ is admittedly a very different animal for this sort of application.
Chuck
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
