github-actions[bot] commented on code in PR #67489:
URL: https://github.com/apache/doris/pull/67489#discussion_r3930786904
##########
fe/fe-core/src/main/java/org/apache/doris/job/extensions/insert/streaming/StreamingInsertTask.java:
##########
@@ -130,12 +157,34 @@ public void run() throws JobException {
}
throw new JobException(errMsg);
} catch (Exception e) {
+ String errorMessage = Util.getRootCauseMessage(e);
+ if (auditEnabled && ctx.getState().getStateType() !=
QueryState.MysqlStateType.ERR) {
+ ctx.getState().setError(ErrorCode.ERR_INTERNAL_ERROR,
errorMessage);
+ }
log.warn("execute insert task error, label is {},offset is {}",
taskCommand.getLabelName(),
runningOffset.toString(), e);
- throw new JobException(Util.getRootCauseMessage(e));
+ throw new JobException(errorMessage);
+ } finally {
+ if (auditEnabled) {
+ AuditLogHelper.logAuditLog(ctx, auditSql,
stmtExecutor.getParsedStmt(),
+ stmtExecutor.getQueryStatisticsForAuditLog(), true);
+ }
}
}
+ private String getAuditSql(TreeMap<Pair<Integer, Integer>, String>
replacements) {
+ List<UnboundTVFRelation> tvfRelations =
taskCommand.getAllTVFRelation();
+ Preconditions.checkState(replacements.size() == 1 &&
tvfRelations.size() == 1,
+ "S3 streaming insert must contain exactly one TVF");
+ String rewrittenProperties =
tvfRelations.get(0).getProperties().getMap().entrySet().stream()
+ .map(entry ->
SqlLiteralUtils.quoteStringLiteral(entry.getKey()) + " = "
+ + SqlLiteralUtils.quoteStringLiteral(entry.getValue()))
+ .collect(Collectors.joining(", "));
+ Pair<Integer, Integer> tvfPropertiesRange = replacements.firstKey();
+ replacements.replace(tvfPropertiesRange, rewrittenProperties);
+ return BaseViewInfo.rewriteSql(replacements, sql);
Review Comment:
[P1] Convert parser offsets before slicing the Java SQL string
`parseForEncryption` records token positions from ANTLR 4.13.1's
`CodePointCharStream`, so these offsets count Unicode code points.
`BaseViewInfo.rewriteSql` applies them directly to a Java `String`, whose
indexes count UTF-16 code units. With a valid key such as
`s3://bucket/input/😀*.csv`, the inclusive stop is one unit early,
reconstruction leaves an extra original quote (`...resolved""`), and the INSERT
can succeed while `AuditLogHelper` catches the reparse failure before
submitting the audit event. Unicode introduced only by the resolved filename
corrupts the second rewrite instead, so please fix the shared slicing boundary:
convert `startCp` and the end-exclusive `stopCp + 1` with
`sql.offsetByCodePoints(...)` (or slice the originating char stream), and add a
test that checks the final event SQL is reparsable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
