[
https://issues.apache.org/jira/browse/GOBBLIN-2168?focusedWorklogId=940975&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-940975
]
ASF GitHub Bot logged work on GOBBLIN-2168:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 30/Oct/24 17:22
Start Date: 30/Oct/24 17:22
Worklog Time Spent: 10m
Work Description: Will-Lo commented on code in PR #4070:
URL: https://github.com/apache/gobblin/pull/4070#discussion_r1823086220
##########
gobblin-data-management/src/main/java/org/apache/gobblin/data/management/trash/TimeBasedSnapshotCleanupPolicy.java:
##########
@@ -41,6 +42,7 @@ public TimeBasedSnapshotCleanupPolicy(Properties props) {
@Override
public boolean shouldDeleteSnapshot(FileStatus snapshot, Trash trash) {
DateTime snapshotTime =
Trash.TRASH_SNAPSHOT_NAME_FORMATTER.parseDateTime(snapshot.getPath().getName());
- return snapshotTime.plusMinutes(this.retentionMinutes).isBeforeNow();
+ // To ensure that the comparison between snapshotTime and the current time
is done in the same time zone
+ return
snapshotTime.plusMinutes(this.retentionMinutes).isBefore(DateTime.now(DateTimeZone.UTC));
Review Comment:
Should there be a configuration to ensure that it is using UTC compared to
the old behavior? You don't want existing retention jobs to have unintended
consequences due to change in behavior that is not explicit
Issue Time Tracking
-------------------
Worklog Id: (was: 940975)
Time Spent: 20m (was: 10m)
> Use isBefore(nowInUTC) to compare snapshots with the current time in UTC.
> -------------------------------------------------------------------------
>
> Key: GOBBLIN-2168
> URL: https://issues.apache.org/jira/browse/GOBBLIN-2168
> Project: Apache Gobblin
> Issue Type: Improvement
> Reporter: Wei-Hsiang Lin
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In existing trash cleaner, the comparison between snapshotTime and the
> current time is NOT done in the same time zone, this leads to longer hard
> deletion time due to time zone difference.
> the fix ensure that the comparison between snapshotTime and the current time
> is done in the same time zone
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
