Another suggestion I'd like to make is bytearray as memory buffer for
the ndarray. An ndarray could just store or extend a bytearray, instead
of having to deal with malloc/free and and the mess that comes with it.
Python takes care of the reference counts for bytearrays, and ctypes
calls the NumPy core DLL. Yes it will not work with older versions of
Python. But for a complete refactoring ("NumPy 3000"), backwards
compatibility should not matter much. (It's easier to backport bytearray
than track down memory leaks in NumPy's core.)
Sturla
Den 10.06.2010 18:48, skrev Sturla Molden:
I have a few radical suggestions:
1. Use ctypes as glue to the core DLL, so we can completely forget
about refcounts and similar mess. Why put manual reference counting
and error handling in the core? It's stupid.
2. The core should be a plain DLL, loadable with ctypes. (I know David
Cournapeau and Robert Kern is going to hate this.) But if Python can
have a custom loader for .pyd files, so can NumPy for it's core DLL.
For ctypes we just need to specify a fully qualified path to the DLL,
which can be read from a config file or whatever.
3. ctypes will take care of all the mess regarding the GIL. Again
there is no need to re-invent the wheel. ctypes.CDLL releases the GIL
when calling into C, and re-acquires before making callbacks to
Python. ctypes.PyDLL keeps the GIL for legacy library code that are
not threadsafe.
ctypes will also make porting to other Python implementations easier
(or even other languages: Ruby, JacaScript) easier. Not to mention
that it will make NumPy impervious to changes in the Python C API.
4. Write the core in C++ or Fortran (95/2003), not C. ANSI C (aka
C89). Use std::vector<> instead of malloc/free for C++, and possibly
templates for generics on various dtypes.
5. Allow OpenMP pragmas in the core. If arrays are above a certain
size, it should switch to multi-threading.
Sturla
Den 10.06.2010 15:26, skrev Charles R Harris:
A few thoughts came to mind while reading the initial writeup.
1) How is the GIL handled in the callbacks.
2) What about error handling? That is tricky to get right, especially
in C and with reference counting.
3) Is there a general policy as to how the reference counting should
be handled in specific functions? That is, who does the reference
incrementing/decrementing?
4) Boost has some reference counted pointers, have you looked at
them? C++ is admittedly a very different animal for this sort of
application.
Chuck
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
