[
https://issues.apache.org/jira/browse/GOBBLIN-1923?focusedWorklogId=883229&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-883229
]
ASF GitHub Bot logged work on GOBBLIN-1923:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 03/Oct/23 22:41
Start Date: 03/Oct/23 22:41
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #3792:
URL: https://github.com/apache/gobblin/pull/3792#discussion_r1344826494
##########
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/api/MysqlMultiActiveLeaseArbiter.java:
##########
@@ -221,6 +237,31 @@ private void initializeConstantsTable() throws IOException
{
}, true);
}
+ /**
+ * Periodically deletes all rows in the table with event_timestamp older
than the retention period defined by config.
+ */
+ private void runRetentionOnArbitrationTable() {
Review Comment:
Small nit but can we add the code for starting a thread within this method?
Due to how it's implemented as an infinite loop we would generally always want
this function to be run asynchronously
Issue Time Tracking
-------------------
Worklog Id: (was: 883229)
Time Spent: 20m (was: 10m)
> Add retention thread for lease arbiter table
> --------------------------------------------
>
> Key: GOBBLIN-1923
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1923
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-service
> Reporter: Urmi Mustafi
> Assignee: Abhishek Tiwari
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add retention to lease arbiter table so it does not grow unbounded. The table
> can be as large as O(number of flows) which may grow so large that
> reading/writing from this table becomes time consuming and slows down our
> throughput of obtaining and evaluating leases for launching flows.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
