Re: Some questions/proposals for the spec (Layout.md)

2016-04-22 Thread Micah Kornfield
, also saying the fixed 64 byte alignment address is used by default; >>> and hard-code the fixed value in source codes (for example, when allocating >>> buffers for primitive arrays, chunk array buffers, and null bitmap buffers). >>> >>> Please help clarify if

Re: Some questions/proposals for the spec (Layout.md)

2016-04-22 Thread Wes McKinney
nd null bitmap buffers). >> >> Please help clarify if I'm not getting you right. Thanks. >> >> Regards, >> Kai >> >> -Original Message- >> From: Micah Kornfield [mailto:emkornfi...@gmail.com] >> Sent: Saturday, April 09, 2016 12:56 PM &

Re: Some questions/proposals for the spec (Layout.md)

2016-04-09 Thread Micah Kornfield
An additional data-point, it looks like Apache Hive also uses one byte for unions: https://github.com/apache/hive/blob/26b5c7b56a4f28ce3eabc0207566cce46b29b558/serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUnion.java On Fri, Apr 8, 2016 at 8:21 PM, Micah Kornfield

Re: Some questions/proposals for the spec (Layout.md)

2016-04-08 Thread Micah Kornfield
Wes McKinney [mailto:w...@cloudera.com] > Sent: Friday, April 08, 2016 11:40 PM > To: dev@arrow.apache.org > Subject: Re: Some questions/proposals for the spec (Layout.md) > > On the SIMD question, it seems AVX is going to 512 bits, so one could even > argue for 64-byte alignme

RE: Some questions/proposals for the spec (Layout.md)

2016-04-08 Thread Zheng, Kai
questions/proposals for the spec (Layout.md) On the SIMD question, it seems AVX is going to 512 bits, so one could even argue for 64-byte alignment as a matter of future-proofing. AVX2 / 256-bit seems fairly widely available nowadays, but it would be great if Todd or any of the hardware folks (e.g