Hi, We have a data structure that stores points in a point cloud (X, Y, Z, attributes) and we have been approached about replacing the current memory store with Arrow. The issue is that the current data store also has a set of pointers (indirection) that allows for things like subsetting and sorting while keeping the data in place. All data is accessed through the indirection table. What people typically want is to export one or more of these data sets specified by the pointers.
My understanding is that Arrow doesn't support such a scheme as the point of the structure is to allow SIMD and other optimizations gained by processing contiguous data. Am I missing something in my reading of the Arrow docs? Does anyone have thoughts/recommendations, or is Arrow just not a good fit for this kind of thing? Thanks, -- Andrew Bell andrew.bell...@gmail.com
