Thank you, sounds just what I need. Will work on this in due time.
> On 1 Sep 2023, at 00:38, Robert Kern <[email protected]> wrote: > > On Thu, Aug 31, 2023 at 3:25 PM Dom Grigonis <[email protected] > <mailto:[email protected]>> wrote: > Hi everyone, > > I am working with shared arrays and their slices. And trying to subclass > ndarray in such way so that they remap to memory on unpickling. I am using > package SharedArray, which doesn’t micro-manage memory locations, but rather > stores the whole map via shm using unique name. One issue I ran into is that > when I pickle & unpickle slice of memory mapped array, I need to store the > spec of the subset so that I can retrieve the same portion. > > The issue I am working on has been briefly discussed on stack: > https://stackoverflow.com/questions/12421770/find-location-of-slice-in-numpy-array > > <https://stackoverflow.com/questions/12421770/find-location-of-slice-in-numpy-array>. > Although I am not sure if it is the exact same one. > > So, in short, is there a way to determine location of a slice in original > array. Ideally, it would be indexing which works for any non-copy > slice/subset possible. > > If you chase the `.base` chain all the way down, you get to a > `SharedArray.map_owner` object with its `.addr` attribute giving the memory > address that's assigned to it in the current process. This will be the same > address that's in the `'data'` key of that first `ndarray` returned from > `SharedArray.create()` that you are making your views from. In your view > `ndarray` (which may be a view of views, with slices, transposes, reshapes, > and arbitrary `stride_tricks` manipulations in between). If you store the > difference between the view `ndarray`'s `'data'` address from the > `map_owner.addr` address, I think that's all you need to recreate the array > in a different process which gets assigned a different memory address for the > same `shm` name. Just add that offset to the `map_owner.addr` and restore the > rest of the information in `__array_interface__`, and I think you should be > good to go. I don't think you'll need to infer or represent the precise path > of Python-level operations (slices, transposes, reshapes, etc.) to which it > got to that point. > > -- > Robert Kern > _______________________________________________ > NumPy-Discussion mailing list -- [email protected] > <mailto:[email protected]> > To unsubscribe send an email to [email protected] > <mailto:[email protected]> > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > <https://mail.python.org/mailman3/lists/numpy-discussion.python.org/> > Member address: [email protected] <mailto:[email protected]>
_______________________________________________ NumPy-Discussion mailing list -- [email protected] To unsubscribe send an email to [email protected] https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: [email protected]
