Re: Set of primitive physical types

2021-09-06 Thread Micah Kornfield
Hi Jorge, > Wrt to the extension type: I am not sure we can make it fast, though: the > interpretation of the bytes would need to be done dynamically (instead of > statically) because we can't compile the struct prior to receiving it (via > IPC or FFI). This interpretation would be part of hot

Re: Set of primitive physical types

2021-09-02 Thread Jorge Cardoso Leitão
Hi, Thanks a lot for the feedback. Atm I was really just trying to get whether others also saw these types as these packed structs. Wrt to the extension type: I am not sure we can make it fast, though: the interpretation of the bytes would need to be done dynamically (instead of statically)

Re: Set of primitive physical types

2021-09-01 Thread Micah Kornfield
I agree, it is what I would have proposed for the interval type if there wasn't an interval type in Arrow already. I think FixedSizeList has for better or worse solved a lot of the problems that a struct type would be used for (e.g. coordinates) Cheers, Micah On Tue, Aug 31, 2021 at 8:27 AM Wes

Re: Set of primitive physical types

2021-08-31 Thread Wes McKinney
I do still think that having a "packed C struct" type would be a useful thing, but thus far no one has needed it enough to develop something in the columnar format specification. On Tue, Aug 31, 2021 at 1:33 AM Micah Kornfield wrote: > > Hi Jorge, > Are there places in the docs that you think

Re: Set of primitive physical types

2021-08-31 Thread Micah Kornfield
Hi Jorge, Are there places in the docs that you think this would simplify? There is an old JIRA [1] about introducing a c-struct type that I think aligns with this observation [1] -Micah [1] https://issues.apache.org/jira/browse/ARROW-1790 On Mon, Aug 30, 2021 at 2:57 PM Jorge Cardoso Leitão

Set of primitive physical types

2021-08-30 Thread Jorge Cardoso Leitão
Hi, Just came across this curiosity that IMO may help us to design physical types in the future. Not sure if this was mentioned before, but it seems to me that `DaysMilliseconds` and `MonthDayNano` belong to a broader class of physical types "typed tuples" in that they are constructed by