github-actions[bot] commented on code in PR #65262:
URL: https://github.com/apache/doris/pull/65262#discussion_r3599878507
##########
fe/be-java-extensions/iceberg-metadata-scanner/src/main/java/org/apache/doris/iceberg/IcebergSysTableJniScanner.java:
##########
@@ -106,9 +109,10 @@ protected int getNext() throws IOException {
break;
}
StructLike row = reader.next();
- for (int i = 0; i < fields.size(); i++) {
- SelectedField field = fields.get(i);
- Object value = row.get(field.sourceIndex,
field.field.type().typeId().javaClass());
+ for (int i = 0; i < requiredFieldCount; i++) {
+ // FE keeps the fields requested by BE at the start of the
Iceberg projection.
+ // FileScanTask.schema() is not the row schema for every
DataTask implementation.
+ Object value = row.get(i, Object.class);
ColumnValue columnValue = new
IcebergSysTableColumnValue(value, timezone);
Review Comment:
[P1] Version the projected-row ordinal contract
This is distinct from the existing zero-column `COUNT(*)` thread: nonempty
projections become an unversioned FE-to-BE row-layout protocol here. With an
old FE, the task is unprojected, so a new BE executing `SELECT snapshot_id FROM
t$snapshots` receives `required_fields=snapshot_id` but reads row 0
(`committed_at`) and can silently return the timestamp as the snapshot ID. The
reverse order also fails: in Iceberg 1.10.1, `StaticDataTask.schema()` remains
the full schema while `rows()` uses the projected layout, so an old scanner
resolves `snapshot_id` to full-schema ordinal 1 and applies it to a one-field
projected row. Both legacy and v2 BE readers instantiate this per-BE Java
scanner, and there is no capability gate, so rolling upgrades can return wrong
data or throw. Please add an explicit version/capability marker with a
backward-compatible fallback, enable FE projection only when target BEs support
it, and cover both upgrade directions with a non-prefix static metadata field.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
