Re: [I] failed to commit txn: MaybeCommitted [doris]
github-actions[bot] closed issue #56154: failed to commit txn: MaybeCommitted URL: https://github.com/apache/doris/issues/56154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] failed to commit txn: MaybeCommitted [doris]
github-actions[bot] commented on issue #56154: URL: https://github.com/apache/doris/issues/56154#issuecomment-4367574935 We're closing this issue because it hasn't been updated in a while. If you'd like to revive this issue, please reopen it and feel free a maintainer to remove the Stale tag! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] failed to commit txn: MaybeCommitted [doris]
atifiu commented on issue #56154: URL: https://github.com/apache/doris/issues/56154#issuecomment-3483691747 We are getting a similar issue and identified that this is caused by parameter `lower_case_table_names` set to non default value of 1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] failed to commit txn: MaybeCommitted [doris]
AItale-4 commented on issue #56154: URL: https://github.com/apache/doris/issues/56154#issuecomment-3316462777 > you can check your fe.audit.log file whether there is more than once `insert overwrite` This error occurs rarely, but it happened again recently. I checked the fe.audit.log on all FE nodes by filtering with the table name, and only one FE node contained related logs. I think it is not related to multi-threaded execution. The log is as follows: ``` 2025-09-22 08:03:54,594 [query] |QueryId=xx|Timestamp=2025-09-22 08:03:44.049|Client=***.***.***.***:41052|User=***|FeIp=***.***.***.***| Ctl=internal|Db=|State=ERR|ErrorCode=1105| ErrorMessage=errCode = 2, detailMessage = internal error, commitTxn failed, transactionId:3945676102130688, code:KV_TXN_COMMIT_ERR, msg:failed to commit kv txn, txn_id=3945676102130688 err=MaybeCommitted| Time(ms)=3875|CpuTimeMS=690|PeakMemoryBytes=185772937|ScanBytes=174661632|ScanRows=592900|ReturnRows=0| ShuffleSendRows=0|ShuffleSendBytes=0|ScanBytesFromLocalStorage=0|ScanBytesFromRemoteStorage=0| FuzzyVariables=|CommandType=Query|StmtType=INSERT|StmtId=23586| SqlHash=89990e0acd1da6e9aac2e6e92f305136|SqlDigest=|IsQuery=false|IsNereids=true| WorkloadGroup=normal|ComputeGroupName=defaultComputeGroup| Stmt=INSERT OVERWRITE TABLE (...) SELECT ... FROM WHERE dt = '20250921' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] failed to commit txn: MaybeCommitted [doris]
ixzc commented on issue #56154: URL: https://github.com/apache/doris/issues/56154#issuecomment-3308922374 you can check your fe.audit.log file whether there is more than once `insert overwrite` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [I] failed to commit txn: MaybeCommitted [doris]
AItale-4 commented on issue #56154:
URL: https://github.com/apache/doris/issues/56154#issuecomment-3305171000
> It is possible that concurrent INSERT OVERWRITE operations on the same
partition by two threads caused one of them to fail. You can check the audit
logs of all FEs around 10:12 on September 17, 2025, with the filter keyword set
to the name of this import table. If there are too many edit logs, add the
keyword "overwrite" for further filtering. If there are conflicting write
operations on the same partition, then this is the root cause of the issue.
I encountered the same error again. Below are the logs from each FE node.
Could you please help me confirm whether the cause is indeed as you described,
or if it might be due to something else? Also, since I only executed a single
INSERT OVERWRITE operation, why would there be concurrent multi-threaded writes?
FE Master
2025-09-17 21:03:37,493 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:37,752 INFO (InsertOverwriteDropDirtyPartitions|348)
[InsertOverwriteManager.runAfterCatalogReady():369] start clean insert
overwrite temp partitions
2025-09-17 21:03:38,011 INFO (thrift-server-pool-126|348)
[CollectRelation.collectFromUnboundRelation():184] collect table [internal,
ads, ads_mes_hr_attendance_attend_check_zh_df] from INSERT_TARGET
2025-09-17 21:03:38,011 INFO (thrift-server-pool-126|348)
[CollectRelation.collectFromUnboundRelation():184] collect table [hive, ads,
ads_mes_hr_attendance_attend_check_df] from QUERY
2025-09-17 21:03:38,013 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.fetchRowCount():617] Will estimate row count for table
ads_mes_hr_attendance_attend_check_df from file list.
2025-09-17 21:03:38,013 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getAllPartitionValues():985] Partition values size for hive
table ads_mes_hr_attendance_attend_check_df is 32
2025-09-17 21:03:38,033 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getFilesForPartitions():1023] Partition list size for hive
partition table ads_mes_hr_attendance_attend_check_df is 30
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():932] Number of files selected for
hive table ads_mes_hr_attendance_attend_check_df is 30
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():957] Table
ads_mes_hr_attendance_attend_check_df sampled 30 of 32 partitions, sampled size
is 334086687
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():962] Table
ads_mes_hr_attendance_attend_check_df rows 1713265, total size is 356359132,
estimatedRowSize is 208
2025-09-17 21:03:38,194 INFO (thrift-server-pool-126|348)
[PropertyAnalyzer.analyzeDataProperty():380] Can not assign cool down timestamp
to HDD storage medium, ignore user setting.
2025-09-17 21:03:38,493 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:39,497 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:40,220 WARN (thrift-server-pool-126|348)
[CloudInternalCatalog.preparePartition():479] preparePartition response: status
{
code: KV_TXN_COMMIT_ERR
msg: "failed to commit txn: MaybeCommitted"
}
2025-09-17 21:03:40,222 WARN (thrift-server-pool-126|348)
[InsertOverwriteTableCommand.run():222] insert into overwrite failed with
task(or group) id 1757751988211
2025-09-17 21:03:40,222 INFO (thrift-server-pool-126|348)
[InsertOverwriteManager.taskFail():214] insert overwrite task [1757751988211]
failed
2025-09-17 21:03:40,222 INFO (thrift-server-pool-126|348)
[InsertOverwriteManager.removeTask():260] remove insert overwrite task:
InsertOverwriteTask{cancel=false, dbId=1754661298419, tableId=1754661299395,
tempPartitionNames=[iot_temp_17647_ads_mes_hr_atte63370384_1758114218014]}
2025-09-17 21:03:40,246 WARN (thrift-server-pool-126|348)
[StmtExecutor.execute():617] Analyze failed. stmt[214338,
37a560c4fd734381-bd494f63a24b8bee]
org.apache.doris.common.NereidsException: errCode = 2, detailMessage =
Failed to ADD PARTITION iot_temp_17647_ads_mes_hr_atte63370384_1758114218014
LIKE ads_mes_hr_attendance_attend_check_zh_df. Reason: errCode = 2,
detailMessage = failed to commit txn: MaybeCommitted
at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.ja
Re: [I] failed to commit txn: MaybeCommitted [doris]
AItale-4 commented on issue #56154:
URL: https://github.com/apache/doris/issues/56154#issuecomment-3305138998
> It is possible that concurrent INSERT OVERWRITE operations on the same
partition by two threads caused one of them to fail. You can check the audit
logs of all FEs around 10:12 on September 17, 2025, with the filter keyword set
to the name of this import table. If there are too many edit logs, add the
keyword "overwrite" for further filtering. If there are conflicting write
operations on the same partition, then this is the root cause of the issue.
I encountered the same error again. Below are the logs from each FE node.
Could you please help me confirm whether the cause is indeed as you described,
or if it might be due to something else? Also, since I only executed a single
INSERT OVERWRITE operation, why would there be concurrent multi-threaded writes?
FE Master
2025-09-17 21:03:37,493 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:37,752 INFO (InsertOverwriteDropDirtyPartitions|348)
[InsertOverwriteManager.runAfterCatalogReady():369] start clean insert
overwrite temp partitions
2025-09-17 21:03:38,011 INFO (thrift-server-pool-126|348)
[CollectRelation.collectFromUnboundRelation():184] collect table [internal,
ads, ads_mes_hr_attendance_attend_check_zh_df] from INSERT_TARGET
2025-09-17 21:03:38,011 INFO (thrift-server-pool-126|348)
[CollectRelation.collectFromUnboundRelation():184] collect table [hive, ads,
ads_mes_hr_attendance_attend_check_df] from QUERY
2025-09-17 21:03:38,013 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.fetchRowCount():617] Will estimate row count for table
ads_mes_hr_attendance_attend_check_df from file list.
2025-09-17 21:03:38,013 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getAllPartitionValues():985] Partition values size for hive
table ads_mes_hr_attendance_attend_check_df is 32
2025-09-17 21:03:38,033 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getFilesForPartitions():1023] Partition list size for hive
partition table ads_mes_hr_attendance_attend_check_df is 30
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():932] Number of files selected for
hive table ads_mes_hr_attendance_attend_check_df is 30
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():957] Table
ads_mes_hr_attendance_attend_check_df sampled 30 of 32 partitions, sampled size
is 334086687
2025-09-17 21:03:38,036 INFO (NotCheckpointRowCountRefreshExecutor-49|348)
[HMSExternalTable.getRowCountFromFileList():962] Table
ads_mes_hr_attendance_attend_check_df rows 1713265, total size is 356359132,
estimatedRowSize is 208
2025-09-17 21:03:38,194 INFO (thrift-server-pool-126|348)
[PropertyAnalyzer.analyzeDataProperty():380] Can not assign cool down timestamp
to HDD storage medium, ignore user setting.
2025-09-17 21:03:38,493 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:39,497 INFO (Statistics Job Appender|348)
[ConnectContext.getCloudCluster():1335] finally set context compute group name
defaultComputeGroup for user 'admin'@'%' with chose way 'authorized cluster'
2025-09-17 21:03:40,220 WARN (thrift-server-pool-126|348)
[CloudInternalCatalog.preparePartition():479] preparePartition response: status
{
code: KV_TXN_COMMIT_ERR
msg: "failed to commit txn: MaybeCommitted"
}
2025-09-17 21:03:40,222 WARN (thrift-server-pool-126|348)
[InsertOverwriteTableCommand.run():222] insert into overwrite failed with
task(or group) id 1757751988211
2025-09-17 21:03:40,222 INFO (thrift-server-pool-126|348)
[InsertOverwriteManager.taskFail():214] insert overwrite task [1757751988211]
failed
2025-09-17 21:03:40,222 INFO (thrift-server-pool-126|348)
[InsertOverwriteManager.removeTask():260] remove insert overwrite task:
InsertOverwriteTask{cancel=false, dbId=1754661298419, tableId=1754661299395,
tempPartitionNames=[iot_temp_17647_ads_mes_hr_atte63370384_1758114218014]}
2025-09-17 21:03:40,246 WARN (thrift-server-pool-126|348)
[StmtExecutor.execute():617] Analyze failed. stmt[214338,
37a560c4fd734381-bd494f63a24b8bee]
org.apache.doris.common.NereidsException: errCode = 2, detailMessage =
Failed to ADD PARTITION iot_temp_17647_ads_mes_hr_atte63370384_1758114218014
LIKE ads_mes_hr_attendance_attend_check_zh_df. Reason: errCode = 2,
detailMessage = failed to commit txn: MaybeCommitted
at
org.apache.doris.qe.StmtExecutor.executeByNerei
Re: [I] failed to commit txn: MaybeCommitted [doris]
ixzc commented on issue #56154: URL: https://github.com/apache/doris/issues/56154#issuecomment-3304321225 It is possible that concurrent INSERT OVERWRITE operations on the same partition by two threads caused one of them to fail. You can check the audit logs of all FEs around 10:12 on September 17, 2025, with the filter keyword set to the name of this import table. If there are too many edit logs, add the keyword "overwrite" for further filtering. If there are conflicting write operations on the same partition, then this is the root cause of the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
