Hi Stefan, One possibility that comes to mind: you may want in any case some way to temporarily "pin" an object's memory in place (e.g., to prevent one thread trying to migrate it while some other thread is working on it). If so then the Python wrapper could acquire a pin when the ndarray is allocated, and release it when it is released. (The canonical way to do this is to create a little opaque Python class that knows how to do the acquire/release, and then assign it to the 'base' attribute of your array -- the semantics of 'base' are simply that ndarray.__del__ will decref whatever object is in 'base'.)
-n On Thu, May 22, 2014 at 12:44 AM, Stefan Seefeld <ste...@seefeld.name> wrote: > Hi Nathaniel, > > thanks for the prompt and thorough answer. You are entirely right, I > hadn't thought things through properly, so let me back up a bit. > > I want to provide Python bindings to a C++ library I'm writing, which is > based on vector/matrix/tensor data types. In my naive view I would > expose these data types as NumPy arrays, creating PyArrayObject > instances as "wrappers", i.e. who borrow raw pointers to the storage > managed by the C++ objects. To make things slightly more interesting, > those C++ objects have their own storage management mechanism, which > allows data to migrate across different address spaces (such as from > host to GPU-device memory), and thus whether the host storage is valid > (i.e., contains up-to-date data) or not depends on where the last > operation was performed (which is controlled by an operation dispatcher > that is part of the library, too). > > It seems if I let Python control the data lifetime, and borrow the data > temporarily from C++ I may be fine. However, I may want to expose > pre-existing C++ objects into Python, though, and it sounds like that > might be dangerous unless I am willing to clone the data so the Python > runtime can hold on to that even after my C++ runtime has released > theirs. But that changes the semantics, as the Python runtime no longer > sees the same data as the C++ runtime, unless I keep the two in sync > each time I cross the language boundary, which may be quite a costly > operation... > > Does all that sound sensible ? > > It seems I have some more design to do. > > Thanks, > Stefan > > -- > > ...ich hab' noch einen Koffer in Berlin... > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
