[
https://issues.apache.org/jira/browse/ARROW-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rok Mihevc updated ARROW-76:
----------------------------
External issue URL: https://github.com/apache/arrow/issues/15495
> Revise format document to include null count, defer non-nullable arrays to
> the domain of metadata
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-76
> URL: https://issues.apache.org/jira/browse/ARROW-76
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Format
> Reporter: Wes McKinney
> Assignee: Wes McKinney
> Priority: Major
> Fix For: 0.1.0
>
>
> After many mailing list discussions and a first draft of metadata
> serialization
> (https://github.com/apache/arrow/blob/master/format/Message.fbs), I propose
> we strike the notion of required fields from the physical memory layout and
> instead make this a property of the metadata (see the {{nullable}} member of
> {{Field}} in the serialized schema metadata).
> Note that Arrow implementations will be free to create specialized containers
> for data having {{nullable = false}} in their metadata. It will be the
> implementations' responsibility to validate that {{null_count == 0}} at IPC
> boundaries when interacting with "untrusted" data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
