Gabriel39 commented on code in PR #68667:
URL: https://github.com/apache/doris/pull/68667#discussion_r4218368755
##########
be/src/format/arrow/arrow_row_batch.cpp:
##########
@@ -229,6 +245,25 @@ std::string
DorisArrowSchemaConvertor::timestamp_timezone(PrimitiveType) const {
return _timezone == "Z" ? "UTC" : _timezone;
}
+Status ArrowFlightSchemaConvertor::convert_to_arrow_type(
+ const DataTypePtr& type, std::shared_ptr<arrow::DataType>* result)
const {
+ // Flight always uses native Variant, including recursively converted
children.
+ if (type->get_primitive_type() == TYPE_VARIANT) {
Review Comment:
Thanks for pointing out the external scanner call path. We have decided to
use the same native Variant V2 representation for both Flight SQL / ADBC and
the external scanner, including MEMORY_SCRATCH_SINK. We will not preserve a
separate implicit UTF8 mapping for the scanner or introduce a protocol opt-in.
This is an intentional change to the external scanner's output contract:
Variant V2 is returned as arrow.parquet.variant with binary metadata/value
storage, and legacy Variant is unsupported. Clients need to handle the native
representation; callers that require text should explicitly CAST the result AS
STRING.
The behavior identified here is therefore accepted as part of the intended
scope, so we will keep the shared mapping and resolve this thread.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
