Yes, an array of references still has a fixed size width in the array buffer. You can think of each entry in the array as a pointer to some other memory on the heap, which can be a dynamic memory allocation.
There's no way in NumPy to support variable-sized array elements in the array buffer, since that assumption is key to how numpy implements strided ufuncs and broadcasting., On Wed, Mar 13, 2024 at 9:34 AM Dom Grigonis <dom.grigo...@gmail.com> wrote: > Thank you for this. > > I am just starting to think about these things, so I appreciate your > patience. > > But isn’t it still true that all elements of an array are still of the > same size in memory? > > I am thinking along the lines of per-element dynamic memory management. > Such that if I had array [0, 1e10000], the first element would default to > reasonably small size in memory. > > On 13 Mar 2024, at 16:29, Nathan <nathan.goldb...@gmail.com> wrote: > > It is possible to do this using the new DType system. > > Sebastian wrote a sketch for a DType backed by the GNU multiprecision > float library: > https://github.com/numpy/numpy-user-dtypes/tree/main/mpfdtype > > It adds a significant amount of complexity to store data outside the array > buffer and introduces the possibility of use-after-free and dangling > reference errors that are impossible if the array does not use embedded > references, so that’s the main reason it hasn’t been done much. > > On Wed, Mar 13, 2024 at 8:17 AM Dom Grigonis <dom.grigo...@gmail.com> > wrote: > >> Hi all, >> >> Say python’s builtin `int` type. It can be as large as memory allows. >> >> np.ndarray on the other hand is optimized for vectorization via strides, >> memory structure and many things that I probably don’t know. Well the point >> is that it is convenient and efficient to use for many things in comparison >> to python’s built-in list of integers. >> >> So, I am thinking whether something in between exists? (And obviously >> something more clever than np.array(dtype=object)) >> >> Probably something similar to `StringDType`, but for integers and >> floats. (It’s just my guess. I don’t know anything about `StringDType`, >> but just guessing it must be better than np.array(dtype=object) in >> combination with np.vectorize) >> >> Regards, >> dgpb >> >> _______________________________________________ >> NumPy-Discussion mailing list -- numpy-discussion@python.org >> To unsubscribe send an email to numpy-discussion-le...@python.org >> https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ >> Member address: nathan12...@gmail.com >> > _______________________________________________ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: dom.grigo...@gmail.com > > > _______________________________________________ > NumPy-Discussion mailing list -- numpy-discussion@python.org > To unsubscribe send an email to numpy-discussion-le...@python.org > https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ > Member address: nathan12...@gmail.com >
_______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
