Re: [Question] Rational for offsets instead of deltas

2021-06-18 Thread Wes McKinney
On Fri, Jun 18, 2021 at 1:12 AM Micah Kornfield wrote: > > > > > Is it to ensure O(1) random access (instead of having to sum all > > deltas up to the index)? > > > This is my understanding of why it was chosen. Yes, that's the reason. For example, certain columnar query processing patterns

Re: [Question] Rational for offsets instead of deltas

2021-06-18 Thread Micah Kornfield
> > Is it to ensure O(1) random access (instead of having to sum all > deltas up to the index)? This is my understanding of why it was chosen. On Thu, Jun 17, 2021 at 10:32 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > Hi, > > (this has no direction; I am just genuinely curious)

[Question] Rational for offsets instead of deltas

2021-06-17 Thread Jorge Cardoso Leitão
Hi, (this has no direction; I am just genuinely curious) I am wondering, what is the rational to use "offsets" instead of "lengths" to represent variable sized arrays? I.e. ["a", "", None, "ab"] is represented as offsets: [0, 1, 1, 1, 3] values: "aab" what is the reasoning to use this over