thorfour commented on code in PR #10230:
URL: https://github.com/apache/arrow-rs/pull/10230#discussion_r3492368644
##########
arrow-ipc/src/writer.rs:
##########
@@ -536,7 +536,17 @@ impl IpcDataGenerator {
compression_context: &mut CompressionContext,
) -> Result<(), ArrowError> {
match column.data_type() {
- DataType::Dictionary(_key_type, _value_type) => {
+ DataType::Dictionary(_key_type, value_type) => {
+ // A dictionary batch writes this field's dictionary values as
its payload.
+ // If that payload is itself dictionary-encoded, readers
cannot match
+ // the encoded buffers to the schema metadata.
+ if matches!(value_type.as_ref(), DataType::Dictionary(_, _)) {
+ return Err(ArrowError::InvalidArgumentError(format!(
+ "IPC does not support dictionary-encoded values for
dictionary field {:?}",
Review Comment:
I'm not overly familiar with the IPC spec but if it could support dict of
dict encoding, can we update this error message to point to an issue about
implementing it in the future?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
