kou commented on code in PR #43950:
URL: https://github.com/apache/arrow/pull/43950#discussion_r1744523130
##########
docs/source/format/Columnar.rst:
##########
@@ -1385,6 +1385,37 @@ have two entries in each RecordBatch. For a RecordBatch
of this schema with
buffer 13: col2 data
+Compression
+-----------
+
+There are three different options for compression of record batch
+body buffers: Buffers can be uncompressed, buffers can be
+compressed with the``lz4`` compression codec, or buffers can
Review Comment:
How about specifying that this ```` ``lz4`` ```` means LZ4 frame format (
https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md ) not LZ4 block
format ( https://github.com/lz4/lz4/blob/dev/doc/lz4_Block_format.md )
explicitly?
See also:
https://github.com/apache/arrow/blob/50219ef69f712ca12fa5c85367a4cae9776d1c99/format/Message.fbs#L46-L49
##########
docs/source/format/Columnar.rst:
##########
@@ -1385,6 +1385,37 @@ have two entries in each RecordBatch. For a RecordBatch
of this schema with
buffer 13: col2 data
+Compression
+-----------
+
+There are three different options for compression of record batch
+body buffers: Buffers can be uncompressed, buffers can be
+compressed with the``lz4`` compression codec, or buffers can
+be compressed with the ``zstd`` compression codec. Buffers in
+the flat sequence of a message body must be either all
+uncompressed or all compressed separately using the same codec.
+
+The difference between compressed and uncompressed buffers in the
+serialized form is as follows:
+
+* If the buffers in the ``RecordBatch`` message are **compressed**
Review Comment:
Can we make the ```` ``RecordBatch`` ```` part as a link to
https://arrow.apache.org/docs/format/Columnar.html#recordbatch-message ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
