github-actions[bot] commented on code in PR #63182:
URL: https://github.com/apache/doris/pull/63182#discussion_r3233861306
##########
fe/fe-core/src/main/java/org/apache/doris/catalog/OlapTable.java:
##########
@@ -3213,8 +3212,33 @@ public void validateForFlexiblePartialUpdate() throws
UserException {
throw new UserException("Flexible partial update can only support
table with light_schema_change enabled."
+ " But table " + getName() + "'s property
light_schema_change is false");
}
- if (hasVariantColumns()) {
- throw new UserException("Flexible partial update can only support
table without variant columns.");
+ validateVariantColumnsForFlexiblePartialUpdate();
+ }
+
+ public void validateVariantColumnsForFlexiblePartialUpdate() throws
UserException {
+ validateVariantColumnsForFlexiblePartialUpdate(getBaseSchema(),
variantEnableFlattenNested());
+ }
+
+ public static void
validateVariantColumnsForFlexiblePartialUpdate(List<Column> columns) throws
UserException {
+ validateVariantColumnsForFlexiblePartialUpdate(columns, false);
Review Comment:
This table-level validator is now shared by stream-load planning, so
relaxing it from "no VARIANT columns" to only doc-mode/flatten checks lets
stream load create VARIANT flexible partial updates without the routine-load
restrictions added in this PR. A concrete path is `STREAM LOAD` with
`unique_key_update_mode=UPDATE_FLEXIBLE_COLUMNS`, `format=json`, and
`jsonpaths` targeting a VARIANT column: `NereidsStreamLoadPlanner` only calls
`destTable.validateForFlexiblePartialUpdate()`, while the BE jsonpath reader
path (`_simdjson_write_columns_by_jsonpath`) does not call the new
flexible-variant write branch and does not append/mark the skip bitmap. The row
can therefore reach storage without VARIANT patch-path markers, and later
duplicate aggregation or publish-conflict replay cannot apply the declared
patch paths correctly. This is distinct from the existing routine-load ALTER
thread because stream load bypasses
`RoutineLoadJob.validateFlexiblePartialUpdateForAlter()` entirely. Pleas
e add equivalent stream-load/load-planning validation (or teach the jsonpath
path to produce skip bitmaps and path markers) and cover it with a negative
regression test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
