mengw15 commented on code in PR #7744:
URL: https://github.com/apache/texera/pull/7744#discussion_r3812761702
##########
amber/src/main/scala/org/apache/texera/web/service/LakekeeperClient.scala:
##########
@@ -126,12 +144,41 @@ class LakekeeperClient(catalogUri: String =
StorageConfig.icebergRESTCatalogUri)
failOn(response.getStatus, response.getBody, s"drop namespace
'$namespace'")
}
}
- val response =
Unirest.delete(s"$managementBase/warehouse/$warehouseId").asString()
- if (response.getStatus != 404) {
- failOn(response.getStatus, response.getBody, "delete warehouse")
+ // The drops above purge each table's data files asynchronously
(Lakekeeper task
+ // queue `tabular_purge`), and Lakekeeper refuses to delete the warehouse
while
+ // any purge is pending — the tasks need the warehouse's storage profile
to reach
+ // S3, so deleting it first would orphan them and leak the files. It
answers 409
+ // WarehouseHasUnfinishedTasks until the queue drains (normally within
seconds),
+ // so ride that out with a bounded retry; every other error, including any
other
+ // 409, still fails immediately. (#7742)
+ var attempt = 0
+ var delay = unfinishedTasksInitialDelayMillis
+ var deleted = false
+ while (!deleted) {
+ val response =
Unirest.delete(s"$managementBase/warehouse/$warehouseId").asString()
+ attempt += 1
+ if (response.getStatus == 404 || (response.getStatus >= 200 &&
response.getStatus < 300)) {
+ deleted = true
+ } else if (
+ isUnfinishedTasksConflict(response.getStatus, response.getBody) &&
+ attempt <= unfinishedTasksRetries
+ ) {
+ Thread.sleep(delay)
+ delay = math.min(delay * 2, unfinishedTasksMaxDelayMillis)
+ } else {
+ failOn(response.getStatus, response.getBody, "delete warehouse")
+ }
Review Comment:
Done — extended `withBackoff` with a defaulted `shouldRetry` predicate and a
`maxDelayMillis` cap (existing call sites unchanged), and replaced the
hand-written loop with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
