pitrou commented on code in PR #37166: URL: https://github.com/apache/arrow/pull/37166#discussion_r1343969434
########## docs/source/format/CanonicalExtensions.rst: ########## @@ -148,6 +148,112 @@ Fixed shape tensor by this specification. Instead, this extension type lets one use fixed shape tensors as elements in a field of a RecordBatch or a Table. +.. _variable_shape_tensor_extension: + +Variable shape tensor +===================== + +* Extension name: `arrow.variable_shape_tensor`. + +* The storage type of the extension is: ``StructArray`` where struct + is composed of **data** and **shape** fields describing a single + tensor per row: + + * **data** is a ``List`` holding tensor elements of a single tensor. + Data type of the list elements is uniform across the entire column. + * **shape** is a ``FixedSizeList<uint32>[ndim]`` of the tensor shape where Review Comment: I have not followed this, but was there any discussion of signed vs. unsigned here? Most of the Arrow spec uses signed integers even for numbers that are always positive (and unless your tensors are huge I'm not sure having one less bit to represent the shape's dimensions is a problem?). ########## docs/source/format/CanonicalExtensions.rst: ########## @@ -148,6 +148,112 @@ Fixed shape tensor by this specification. Instead, this extension type lets one use fixed shape tensors as elements in a field of a RecordBatch or a Table. +.. _variable_shape_tensor_extension: + +Variable shape tensor +===================== + +* Extension name: `arrow.variable_shape_tensor`. + +* The storage type of the extension is: ``StructArray`` where struct + is composed of **data** and **shape** fields describing a single + tensor per row: + + * **data** is a ``List`` holding tensor elements of a single tensor. + Data type of the list elements is uniform across the entire column. + * **shape** is a ``FixedSizeList<uint32>[ndim]`` of the tensor shape where Review Comment: I have not followed this, but was there any discussion of signed vs. unsigned here? Most of the Arrows spec use signed integers even for numbers that are always positive (and unless your tensors are huge I'm not sure having one less bit to represent the shape's dimensions is a problem?). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
