martin-g commented on code in PR #20063:
URL: https://github.com/apache/datafusion/pull/20063#discussion_r2746019284
##########
datafusion/proto-common/src/from_proto/mod.rs:
##########
@@ -406,6 +411,35 @@ impl TryFrom<&protobuf::ScalarValue> for ScalarValue {
));
};
+ // IPC dictionary batch IDs are assigned when encoding the
schema, but our protobuf
+ // `Schema` doesn't preserve those IDs. Reconstruct them
deterministically by
+ // round-tripping the schema through IPC.
+ let schema: Schema = {
+ let ipc_gen = IpcDataGenerator {};
+ let write_options = IpcWriteOptions::default();
+ let mut dict_tracker = DictionaryTracker::new(false);
+ let encoded_schema =
ipc_gen.schema_to_bytes_with_dictionary_tracker(
+ &schema,
+ &mut dict_tracker,
+ &write_options,
+ );
+ let message =
+
root_as_message(encoded_schema.ipc_message.as_slice()).map_err(
+ |e| {
+ Error::General(format!(
+ "Error IPC schema message while
deserializing ScalarValue::List: {e}"
Review Comment:
Why the error messages say `ScalarValue::List` ? (**List**)
Isn't this used for any nested type ? List, Map, Struct, ...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
