github-actions[bot] commented on code in PR #65127:
URL: https://github.com/apache/doris/pull/65127#discussion_r3530027380
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/load/NereidsDataDescription.java:
##########
@@ -990,9 +990,10 @@ private void checkLoadPriv(String fullDbName) throws
AnalysisException {
}
}
- // Change all the columns name to lower case, because Doris column is
case-insensitive.
+ // Change text-format column names to lower case, because Doris column is
case-insensitive.
private void columnsNameToLowerCase(List<String> columns) {
- if (columns == null || columns.isEmpty() ||
"json".equals(analysisMap.get(FileFormatProperties.PROP_FORMAT))) {
+ String format = analysisMap.get(FileFormatProperties.PROP_FORMAT);
+ if (columns == null || columns.isEmpty() ||
Util.isCasePreservingFormat(format)) {
Review Comment:
This helper is also used for `columnsFromPath` via the constructor calls
above, and the legacy `DataDescription` helper was changed the same way, but
Arrow's case-sensitive requirement only applies to file schema field names.
Path-derived columns are matched later against partition directory keys:
`NereidsFileGroupInfo` calls
`FilePartitionUtils.parseColumnsFromPathWithNullInfo(..., true, false)`, and
that parser compares the expected name to the segment before `=` exactly. So a
broker load like `FORMAT AS arrow ... COLUMNS FROM PATH AS (DT)` over a normal
path `.../dt=2026/file.arrow` used to lower `DT` to `dt`, but now this guard
preserves `DT` and the path parser fails with `Fail to parse columnsFromPath`.
Please keep `columnsFromPath` on the old canonicalization path, or make
path-key extraction intentionally canonical/case-insensitive while preserving
only actual Arrow source field names.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
