github-actions[bot] commented on code in PR #65553:
URL: https://github.com/apache/doris/pull/65553#discussion_r3572426071
##########
fe/fe-core/src/main/java/org/apache/doris/load/routineload/RLTaskTxnCommitAttachment.java:
##########
@@ -75,6 +76,9 @@ public RLTaskTxnCommitAttachment(TRLTaskTxnCommitAttachment
rlTaskTxnCommitAttac
if (rlTaskTxnCommitAttachment.isSetErrorLogUrl()) {
this.errorLogUrl = rlTaskTxnCommitAttachment.getErrorLogUrl();
}
+ if (rlTaskTxnCommitAttachment.isSetFirstErrorMsg()) {
+ this.firstErrorMsg = rlTaskTxnCommitAttachment.getFirstErrorMsg();
Review Comment:
`RuntimeState::append_error_msg_to_file()` stores `_first_error_msg` as the
full source line before the error-log-file truncation, and this constructor
then keeps that raw string in the transaction attachment. From there it is
persisted with `TransactionState.txnCommitAttachment` and forwarded to cloud
progress by `TxnUtil.rlTaskTxnCommitAttachmentToPb()`; the only
`Config.first_error_msg_max_length` cap happens later in
`RoutineLoadJob.updateFirstErrorMsg()`, after the durable attachment/progress
boundary. A single malformed row with a very large source line can therefore
bloat FE edit logs and cloud metadata even though the displayed value is
shortened. Please cap the value before storing it in
`RLTaskTxnCommitAttachment` or before converting it to the cloud protobuf, and
add coverage for an overlong bad row.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
