> > It would still be desirable to maintain the memory layout of C/C++/NumPy to > maintain zero-copy. > FixedList[2] maintains this layout, while a Struct[re, im] does not.
I noted this before but there are some gaps in Parquet support for FixedSizeList around null handling. Just something to be aware of if this is use-case that we think will be encountered. On Mon, Jun 14, 2021 at 2:02 AM Antoine Pitrou <anto...@python.org> wrote: > > Le 14/06/2021 à 10:54, Simon Perkins a écrit : > > > The reason why I am being nit-picky here is I think that having a > first > > class type indicates that it should eventually be supported by all > > reference implementations. An "well known" extension type I think offers > > less guarantees which makes it seem more suitable for niche types. > > > > What are the requirements imposed on downstream projects by adding new > types > > such as Complex Numbers and Intervals? Hypothetically, does a new > > first-class > > type impose a requirement to provide full support for it downstream? > > This is not a requirement for downstream projects (for example a > dataframe implementation or a database connector), but for Arrow > implementations, that is (mostly) the code that lives in the Apache > Arrow repository. There are a number of them: > https://arrow.apache.org/docs/status.html > > > Or, does adding a type simply involve exposing a new Arrow Type (the > > representation) > > in the respective language (C++/Java/Rust) that downstream projects may > > choose to support or ignore? > > Yes. > > > Java/Rust may not have a native Complex Type for example, but this isn't > > Arrow's responsibility -- it simply provides it's own > > type that the language/project should interpret. > > Having a native complex type isn't a requirement. A complex type is > trivially expressed in (almost?) any language as a struct, anyway. > > Regards > > Antoine. >
