morningman commented on code in PR #69488:
URL: https://github.com/apache/doris/pull/69488#discussion_r4357863339
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java:
##########
@@ -476,15 +489,62 @@ private static Plan normalizePlanWithoutLock(LogicalPlan
plan, TableIf table,
}
static List<Column> connectorWriteSchema(TableIf table, boolean full) {
+ return pinnedConnectorWriteSchema(table).orElseGet(() ->
table.getBaseSchema(full));
+ }
+
+ private static Optional<List<Column>> pinnedConnectorWriteSchema(TableIf
table) {
ConnectContext context = ConnectContext.get();
- if (context != null && context.getStatementContext() != null) {
- Optional<List<Column>> pinned =
-
context.getStatementContext().getConnectorWriteSchema(table.getId());
- if (pinned.isPresent()) {
- return pinned.get();
- }
+ if (context == null || context.getStatementContext() == null) {
+ return Optional.empty();
+ }
+ return
context.getStatementContext().getConnectorWriteSchema(table.getId());
+ }
+
+ /**
+ * The pinned writer schema of a target whose column defaults apply to
writes only. The read schema of such a
+ * table carries no defaults, so DEFAULT(column) in an INSERT into it must
resolve against this schema.
+ */
+ private static Optional<List<Column>> writeOnlyDefaultColumns(TableIf
table) {
+ if (!(table instanceof PluginDrivenExternalTable)
+ || !((PluginDrivenExternalTable)
table).hasWriteOnlyDefaults()) {
+ return Optional.empty();
}
- return table.getBaseSchema(full);
+ return pinnedConnectorWriteSchema(table);
+ }
+
+ /** Resolve DEFAULT(column) in one inserted value by name against the
target's pinned writer schema. */
+ private static NamedExpression resolveDefaultReferences(NamedExpression
value,
+ Optional<List<Column>> writeOnlyDefaultColumns, TableIf table,
List<String> targetNameParts) {
+ if (!writeOnlyDefaultColumns.isPresent()) {
+ return value;
+ }
+ return (NamedExpression)
ConnectorWriteSchemaUtils.resolveDefaultReferences(value,
Review Comment:
Fixed in cf75568b69b. `ConnectorWriteSchemaUtils.resolveDefaultReferences`
now casts each resolved default to the type of the writer column it references
(`TypeCoercionUtils.castUnbound`) before the expression around it is bound, so
`bit_count(DEFAULT(c))` on an INT column with write-default -1 gets an INT -1
in VALUES and in SELECT. The external UPDATE / MERGE / changelog paths use the
same helper and get the same typing. branch-4.2 is not affected: it builds
write defaults as literals of the column type. Test:
`InsertUtilsTest#writeOnlyDefaultIsTypedAsItsColumnBeforeBinding` (VALUES and
SELECT); with the previous commit the argument of bit_count is
`TinyIntLiteral(-1)`.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/expression/rules/RewriteDefaultExpression.java:
##########
@@ -82,6 +87,15 @@ private static Expression
rewrite(ExpressionMatchingContext<Default> context) {
+ column.getName() + "'");
}
+ Optional<TableIf> originalTable = slotRef.getOriginalTable();
+ if (originalTable.filter(PluginDrivenExternalTable.class::isInstance)
+ .filter(table -> ((PluginDrivenExternalTable)
table).hasWriteOnlyDefaults())
+ .isPresent()) {
+ column =
pinnedWriteColumn(context.cascadesContext.getStatementContext(),
originalTable.get(), column)
Review Comment:
Fixed in cf75568b69b. `RewriteDefaultExpression` now takes the result type
from the pinned writer column, after the substitution. Test:
`RewriteDefaultExpressionTest#writeOnlyDefaultTakesTheTypeOfThePinnedWriterColumn`
pins a BIGINT writer column with default 4900000001 over an INT read column
with the same field id and expects a BIGINT literal; with the previous commit
the result is `cast(4900000001 as INT)`.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java:
##########
@@ -476,15 +489,62 @@ private static Plan normalizePlanWithoutLock(LogicalPlan
plan, TableIf table,
}
static List<Column> connectorWriteSchema(TableIf table, boolean full) {
+ return pinnedConnectorWriteSchema(table).orElseGet(() ->
table.getBaseSchema(full));
+ }
+
+ private static Optional<List<Column>> pinnedConnectorWriteSchema(TableIf
table) {
ConnectContext context = ConnectContext.get();
- if (context != null && context.getStatementContext() != null) {
- Optional<List<Column>> pinned =
-
context.getStatementContext().getConnectorWriteSchema(table.getId());
- if (pinned.isPresent()) {
- return pinned.get();
- }
+ if (context == null || context.getStatementContext() == null) {
+ return Optional.empty();
+ }
+ return
context.getStatementContext().getConnectorWriteSchema(table.getId());
+ }
+
+ /**
+ * The pinned writer schema of a target whose column defaults apply to
writes only. The read schema of such a
+ * table carries no defaults, so DEFAULT(column) in an INSERT into it must
resolve against this schema.
+ */
+ private static Optional<List<Column>> writeOnlyDefaultColumns(TableIf
table) {
+ if (!(table instanceof PluginDrivenExternalTable)
+ || !((PluginDrivenExternalTable)
table).hasWriteOnlyDefaults()) {
+ return Optional.empty();
}
- return table.getBaseSchema(full);
+ return pinnedConnectorWriteSchema(table);
+ }
+
+ /** Resolve DEFAULT(column) in one inserted value by name against the
target's pinned writer schema. */
+ private static NamedExpression resolveDefaultReferences(NamedExpression
value,
+ Optional<List<Column>> writeOnlyDefaultColumns, TableIf table,
List<String> targetNameParts) {
+ if (!writeOnlyDefaultColumns.isPresent()) {
+ return value;
+ }
+ return (NamedExpression)
ConnectorWriteSchemaUtils.resolveDefaultReferences(value,
+ writeOnlyDefaultColumns.get(), (ExternalTable) table,
ConnectContext.get(), targetNameParts, null);
+ }
+
+ /**
+ * Resolve DEFAULT(column) in the select lists of an INSERT ... SELECT by
name against the target's pinned
+ * writer schema, before the source relations are bound, so it names a
target column whatever the source is.
+ */
+ private static Plan resolveSelectDefaultReferences(Plan query,
Optional<List<Column>> writeOnlyDefaultColumns,
+ TableIf table, List<String> targetNameParts) {
+ return query.rewriteUp(plan -> {
Review Comment:
Not changing this in this PR. The INSERT ... SELECT resolution covers the
select lists of the main query (projects and the one-row relation), the same
shapes as branch-4.2, which this PR ports
(`InsertUtils#resolveIcebergSelectDefaultReferences` there). In an aggregate
output, a WITH clause or a subquery, DEFAULT(c) keeps its meaning from before
this PR and names a column of its own FROM clause, as in MySQL: with no such
column the statement fails with an unknown-column error; a column of the target
table itself gets the pinned writer default through the bound rewrite; a column
of another Iceberg table now fails with "must reference a column of the table
that this statement writes" instead of answering NULL; a column of a Hive or
OLAP source gets that column's default, as before. Resolving target names in
those shapes needs other analysis stages (the command-level WITH is attached
after normalizePlan; CTE and subquery plans are not children of the project),
so it would be a separa
te change.
##########
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/commands/insert/InsertUtils.java:
##########
@@ -476,15 +489,62 @@ private static Plan normalizePlanWithoutLock(LogicalPlan
plan, TableIf table,
}
static List<Column> connectorWriteSchema(TableIf table, boolean full) {
+ return pinnedConnectorWriteSchema(table).orElseGet(() ->
table.getBaseSchema(full));
+ }
+
+ private static Optional<List<Column>> pinnedConnectorWriteSchema(TableIf
table) {
ConnectContext context = ConnectContext.get();
- if (context != null && context.getStatementContext() != null) {
- Optional<List<Column>> pinned =
-
context.getStatementContext().getConnectorWriteSchema(table.getId());
- if (pinned.isPresent()) {
- return pinned.get();
- }
+ if (context == null || context.getStatementContext() == null) {
+ return Optional.empty();
+ }
+ return
context.getStatementContext().getConnectorWriteSchema(table.getId());
+ }
+
+ /**
+ * The pinned writer schema of a target whose column defaults apply to
writes only. The read schema of such a
+ * table carries no defaults, so DEFAULT(column) in an INSERT into it must
resolve against this schema.
+ */
+ private static Optional<List<Column>> writeOnlyDefaultColumns(TableIf
table) {
+ if (!(table instanceof PluginDrivenExternalTable)
+ || !((PluginDrivenExternalTable)
table).hasWriteOnlyDefaults()) {
+ return Optional.empty();
}
- return table.getBaseSchema(full);
+ return pinnedConnectorWriteSchema(table);
+ }
+
+ /** Resolve DEFAULT(column) in one inserted value by name against the
target's pinned writer schema. */
+ private static NamedExpression resolveDefaultReferences(NamedExpression
value,
+ Optional<List<Column>> writeOnlyDefaultColumns, TableIf table,
List<String> targetNameParts) {
+ if (!writeOnlyDefaultColumns.isPresent()) {
+ return value;
+ }
+ return (NamedExpression)
ConnectorWriteSchemaUtils.resolveDefaultReferences(value,
+ writeOnlyDefaultColumns.get(), (ExternalTable) table,
ConnectContext.get(), targetNameParts, null);
Review Comment:
Not changing this here. DEFAULT(...) in VALUES / SELECT accepts the
qualifications that the existing Iceberg UPDATE / MERGE DEFAULT handling
accepts, through the same `UpdateCommand.checkAssignmentColumn` (branch-4.2
does the same): `c`, `t.c`, `db.t.c`. `ctl.db.t.c` fails with "column in
assignment list is invalid" instead of resolving; it does not write a wrong
value. Accepting catalog-qualified names would change the assignment-column
rule of all DML paths, so it belongs in its own change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
