Gabriel39 commented on code in PR #67921:
URL: https://github.com/apache/doris/pull/67921#discussion_r4011043150
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/lance/LanceTypeConverter.java:
##########
@@ -89,7 +84,8 @@ private static Type toDorisType(Field field, boolean
allowNull) {
ArrowType arrowType = field.getType();
switch (arrowType.getTypeID()) {
case Null:
- return allowNull ? Type.NULL : Type.UNSUPPORTED;
+ // Required Null leaves cannot use the nullable SerDe that
handles Arrow NA buffers.
+ return field.isNullable() ? Type.NULL : Type.UNSUPPORTED;
Review Comment:
Fixed in 116f42eb03. FileTableValuedFunction now delegates
requiresCurrentLanceReader() to the TVF that owns the discovered schema. The
new generic-file S3 test reproduces the missing rejection before the fix,
rejects a nested Null projection on a smooth-upgrade source BE afterward, and
verifies that an ordinary projection remains usable.
##########
be/src/core/data_type/data_type.h:
##########
@@ -170,7 +170,9 @@ class IDataType : private boost::noncopyable {
auto node = ptype->add_types();
node->set_type(TTypeNodeType::SCALAR);
auto scalar_type = node->mutable_scalar_type();
- scalar_type->set_type(doris::to_thrift(get_primitive_type()));
+ // NULL uses UInt8 internally; preserve its logical type even inside
complex schemas.
+ scalar_type->set_type(is_null_literal() ? TPrimitiveType::NULL_TYPE
Review Comment:
Fixed in 116f42eb03. Backend execution affinity is now an
ExternalFileTableValuedFunction contract, forwarded by FileTableValuedFunction
and consumed by TVFScanNode without an instanceof Local check. The new
generic-file shared Local test reproduces distributed scheduling before the fix
and verifies schema-backend pinning afterward. It also checks that shared
non-Lance files retain distributed execution. All 13 tests in TVFScanNodeTest
and ExternalFileTableValuedFunctionTest passed in an isolated runner using
cached dependencies; FE Checkstyle passed. Full build validation is delegated
to CI.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
