> The two last items are really gratuitous limitations. Readers are free > to implement limitations of their own and refuse to read more general > fixed-size-list data, but it does not make sense to impose those > limitations in the spec.
The reason for proposing limitations is that this proposal aims for a narrow vector use case (vector databases, embeddings). In that context non-finite or nested values are invalid. Vector stores typically reject such values. I don't expect applications that need multidimensional array data to use this vector logical type and I don't think it materially changes the spec outside of the new vector logical type. For general fixed-size-list data, I expect we will continue the discussion on new physical layout possibilities and provide a general solution there. > (especially the "non-null and finite" item really belongs to statistics > IMHO) Non-null is already communicated by the schema. Regarding finiteness, statistics are sometimes ignored (not written or read) and require reading metadata, which would require consumers of such type to be more defensive than necessary. (finiteness here means not-nan, not-null, not-+/-inf) I don't think it's too unusual for a narrow logical type to restrict its values. For instance, JSON requires valid JSON, GEOGRAPHY bounds longitudes and latitudes. If we are concerned that writers would have to validate their data, we could add an explicit property `optional bool VectorType.finite`. Thank you for reviewing Antoine! Best, Rok
