Hi Anja,
I believe this was clarified in [1] with the addition of the following
text to the standard
> The buffer pointers MAY be null only in two situations:
>
> 1. for the null bitmap buffer, if :c:member:`ArrowArray.null_count`
is 0;
> 2. for any buffer, if the size in bytes of the corresponding buffer
would be 0.
This is supported by arrow-rs [2], and I believe arrow2 is incorrect in
not supporting this. I would recommend taking this up with the arrow2
maintainers [3].
Kind Regards,
Raphael Taylor-Davies
[1]: https://github.com/apache/arrow/pull/14808
[2]: https://github.com/apache/arrow-rs/pull/3276
[3]: https://github.com/jorgecarleitao/arrow2/pull/1476
On 02/08/2023 17:43, Anja wrote:
Hi!
I opened two issues recently on a spec incompatibility between arrow2 and
arrow-C++: https://github.com/apache/arrow/issues/36960 and
.
Namely, there are situations where arrow-C++ will legally create a null
buffer, and arrow2 does not allow null buffers.
I am happy to provide code-effort, but I am looking for consensus from the
community on this issue. I think an interoperability issue like this is
important to resolve, especially with the plans to unify arrow2 and
arrow-rs: https://github.com/apache/arrow-rs/issues/1176
The `kNonNullFiller` (
https://github.com/apache/arrow/blob/a06b2618420ef89431373a9e8f07a5da64d546a5/cpp/src/arrow/util/ubsan.h#L33)
exists for being used in cases where null buffers can cause issues. I am
proposing changes such that it gets pointed to for representing the reading
of an empty table.
Are there any thoughts? I welcome engagement on either of the issues. This
problem can, in theory, be solved within either project.
~Anja
