Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
anmolnar merged PR #7007: URL: https://github.com/apache/hbase/pull/7007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on PR #7007: URL: https://github.com/apache/hbase/pull/7007#issuecomment-2961170113 @anmolnar all the tests are incremental backup related. we are trying fix those tests separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2960053524
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 23s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 20s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 14s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 6m 1s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 13s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 54s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 22m 20s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 43m 46s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 4e665ef9b5fe 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / e3c24f04e64a256238faa09468140e2a8fb4da0b |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/testReport/
|
| Max. process+thread count | 3293 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2960016859
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 3s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 29s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 28s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 43s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 1s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed |
| -0 :warning: | javac | 0m 29s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 5 new + 109 unchanged - 0 fixed = 114 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | xmllint | 0m 0s | | No new issues. |
| +1 :green_heart: | spotbugs | 0m 36s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 11m 45s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | | The patch does not
generate ASF License warnings. |
| | | 29m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac codespell detsecrets xmllint
hadoopcheck spotless compile spotbugs checkstyle hbaseanti |
| uname | Linux 09aa76675c8b 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / e3c24f04e64a256238faa09468140e2a8fb4da0b |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/7/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
anmolnar commented on code in PR #7007: URL: https://github.com/apache/hbase/pull/7007#discussion_r2138321489 ## hbase-backup/pom.xml: ## @@ -163,6 +163,12 @@ junit test + + org.mockito + mockito-inline + 4.11.0 Review Comment: You don't need to set version here, because it's already in the top level pom. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2958032688
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 14s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 19s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 14s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 5m 59s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 5s | | the patch passed |
| +1 :green_heart: | compile | 0m 21s | | the patch passed |
| +1 :green_heart: | javac | 0m 21s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 15s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 55s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 23m 5s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 44m 0s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit shadedjars compile |
| uname | Linux 675f87ee250d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 73f0e1c176087fd2fc0da6a40a4ad4c3bed53f77 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/testReport/
|
| Max. process+thread count | 3189 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2957997475
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 44s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 32s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 10s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 34s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed |
| -0 :warning: | javac | 0m 29s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 5 new + 109 unchanged - 0 fixed = 114 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | xmllint | 0m 0s | | No new issues. |
| +1 :green_heart: | spotbugs | 0m 36s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 3s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 43s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 9s | | The patch does not
generate ASF License warnings. |
| | | 31m 5s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac codespell detsecrets xmllint
hadoopcheck spotless compile spotbugs checkstyle hbaseanti |
| uname | Linux 16ef2034c40f 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 73f0e1c176087fd2fc0da6a40a4ad4c3bed53f77 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/6/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on PR #7007: URL: https://github.com/apache/hbase/pull/7007#issuecomment-2943439294 > Thanks @vinayakphegde, patch looks good to me. However I have the same criticism that I mentioned previously: unit tests are missing. > > Since all of your helper methods are private you cannot test them individually, so you need to set up an entire starship in your test case, call the command and verify the output. This is end 2 end testing. You will get a yes/no answer to your question about whether my function is working. If the answer is yes, we're fine, but if it's no, you'll have no idea about where the problem is and you have to debug. > > Unit testing individual methods gives more detail about what's working and what's not. Sure @anmolnar I will try to add more unit tests to cover these methods. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
kgeisz commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2127536048
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,139 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
Review Comment:
Hey @vinayakphegde, this is the function that led me to ask for
clarification on why we need to update the start times of the continuous
backups. Maybe you could add another line or two to the docstring here that
elaborates on why we need to do this? That may make it more clear to others in
the future.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
anmolnar commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2127449678
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
+BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
+ throws IOException {
+
+ Map backupTables =
sysTable.getContinuousBackupTableSet();
+ Set tablesToUpdate = new HashSet<>();
+
+ for (Map.Entry entry : backupTables.entrySet()) {
+if (entry.getValue() < cutoffTimestamp) {
+ tablesToUpdate.add(entry.getKey());
+}
+ }
+
+ if (!tablesToUpdate.isEmpty()) {
+sysTable.updateContinuousBackupTableSet(tablesToUpdate,
cutoffTimestamp);
+ }
+}
+
+/**
+ * Cleans up old WAL and bulk-loaded files based on the determined cutoff
timestamp.
+ */
+private void deleteOldWALFiles(Configuration conf, String backupWalDir,
long cutoffTime)
+ throws IOException {
+ System.out.println("Starting WAL cleanup in backup directory: " +
backupWalDir
++ " with cutoff time: " + cutoffTime);
+
+ BackupFileSystemManager manager =
+new BackupFileSystemManager(CONTINUOUS_BACKUP_REPLICATION_PEER, conf,
backupWalDir);
+ FileSystem fs = manager.getBackupFs();
+ Path walDir = manager.getWalsDir();
+ Path bulkloadDir = manager.getBulkLoadFilesDir();
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat(DATE_FORMAT);
+ dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
+
+ System.out.println("Listing directories under: " + walDir);
+
+ FileStatus[] directories = fs.listStatus(walDir);
+
+ for (FileStatus dirStatus : directories) {
+if (!dirStatus.isDirectory()) {
+ continue; // Skip files, we only want directories
+}
+
+Path dirPath = dirStatus.getPath();
+String dirName = dirPath.getName();
+
+try {
+ long dayStart = parseDayDirectory(dirName, dateFormat);
+ System.out
+.println("Checking WAL directory: " + dirName + " (Start Time: " +
dayStart + ")");
+
+ // If WAL files of that day are older than cutoff time, delete them
+ if (dayStart + ONE_DAY_IN_MILLISECONDS - 1 < cutoffTime) {
+System.out.println("Deleting outdated WAL directory: " + dirPath);
+fs.delete(dirPath, true);
Review Comment:
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2941247452
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 48s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 4s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 27s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 37s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 32s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 8m 18s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 17s | | the patch passed |
| +1 :green_heart: | compile | 0m 34s | | the patch passed |
| +1 :green_heart: | javac | 0m 34s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 25s | | the patch passed |
| +1 :green_heart: | shadedjars | 8m 34s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 25m 26s | | hbase-backup in the patch
passed. |
| | | 55m 1s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 2cf9fb5e70ef 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 3655d484e60e3534c7aac8f89f0e4ed0de22bfe3 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/testReport/
|
| Max. process+thread count | 3543 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2941100443
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 43s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 2s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 33s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 11s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 34s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 1s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed |
| -0 :warning: | javac | 0m 30s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 4 new + 109 unchanged - 0 fixed = 113 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 36s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 5s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 31m 53s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 3f85f005e67d 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 3655d484e60e3534c7aac8f89f0e4ed0de22bfe3 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/5/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2126812526
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java:
##
@@ -1056,6 +1056,27 @@ public void addContinuousBackupTableSet(Set
tables, long startTimesta
}
}
+ /**
+ * Updates the system table with the new start timestamps for continuous
backup tables.
+ * @param tablesToUpdateThe set of tables that need their start
timestamps updated.
+ * @param newStartTimestamp The new start timestamp to be set.
+ */
+ public void updateContinuousBackupTableSet(Set tablesToUpdate,
long newStartTimestamp)
+throws IOException {
+try (Table table = connection.getTable(tableName)) {
Review Comment:
NIT: Add a null check for tablesToUpdate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2939041656
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 36s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 53s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 33s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 10s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 35s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 47s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 1s | | the patch passed |
| +1 :green_heart: | compile | 0m 30s | | the patch passed |
| -0 :warning: | javac | 0m 30s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 4 new + 109 unchanged - 0 fixed = 113 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 36s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 4s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 46s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 31m 15s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 2bf02f69872a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 86ba7e2461c1c8abc0cf1da62f4d5b060b5b8627 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2939076252
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 32s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 19s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 15s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 6m 0s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 9s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 13s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 56s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 21m 44s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 42m 51s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 64b93136c57a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 86ba7e2461c1c8abc0cf1da62f4d5b060b5b8627 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/testReport/
|
| Max. process+thread count | 3619 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/4/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125880412
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
+BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
+ throws IOException {
+
+ Map backupTables =
sysTable.getContinuousBackupTableSet();
+ Set tablesToUpdate = new HashSet<>();
+
+ for (Map.Entry entry : backupTables.entrySet()) {
+if (entry.getValue() < cutoffTimestamp) {
+ tablesToUpdate.add(entry.getKey());
+}
+ }
+
+ if (!tablesToUpdate.isEmpty()) {
+sysTable.updateContinuousBackupTableSet(tablesToUpdate,
cutoffTimestamp);
+ }
+}
+
+/**
+ * Cleans up old WAL and bulk-loaded files based on the determined cutoff
timestamp.
+ */
+private void deleteOldWALFiles(Configuration conf, String backupWalDir,
long cutoffTime)
+ throws IOException {
+ System.out.println("Starting WAL cleanup in backup directory: " +
backupWalDir
++ " with cutoff time: " + cutoffTime);
+
+ BackupFileSystemManager manager =
+new BackupFileSystemManager(CONTINUOUS_BACKUP_REPLICATION_PEER, conf,
backupWalDir);
+ FileSystem fs = manager.getBackupFs();
+ Path walDir = manager.getWalsDir();
+ Path bulkloadDir = manager.getBulkLoadFilesDir();
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat(DATE_FORMAT);
+ dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
+
+ System.out.println("Listing directories under: " + walDir);
+
+ FileStatus[] directories = fs.listStatus(walDir);
+
+ for (FileStatus dirStatus : directories) {
+if (!dirStatus.isDirectory()) {
+ continue; // Skip files, we only want directories
+}
+
+Path dirPath = dirStatus.getPath();
+String dirName = dirPath.getName();
+
+try {
+ long dayStart = parseDayDirectory(dirName, dateFormat);
+ System.out
+.println("Checking WAL directory: " + dirName + " (Start Time: " +
dayStart + ")");
+
+ // If WAL files of that day are older than cutoff time, delete them
+ if (dayStart + ONE_DAY_IN_MILLISECONDS - 1 < cutoffTime) {
+System.out.println("Deleting outdated WAL directory: " + dirPath);
+fs.delete(dirPath, true);
Review Comment
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125815526
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
+BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
+ throws IOException {
+
+ Map backupTables =
sysTable.getContinuousBackupTableSet();
+ Set tablesToUpdate = new HashSet<>();
+
+ for (Map.Entry entry : backupTables.entrySet()) {
+if (entry.getValue() < cutoffTimestamp) {
+ tablesToUpdate.add(entry.getKey());
+}
+ }
+
+ if (!tablesToUpdate.isEmpty()) {
+sysTable.updateContinuousBackupTableSet(tablesToUpdate,
cutoffTimestamp);
+ }
+}
+
+/**
+ * Cleans up old WAL and bulk-loaded files based on the determined cutoff
timestamp.
+ */
+private void deleteOldWALFiles(Configuration conf, String backupWalDir,
long cutoffTime)
+ throws IOException {
+ System.out.println("Starting WAL cleanup in backup directory: " +
backupWalDir
++ " with cutoff time: " + cutoffTime);
+
+ BackupFileSystemManager manager =
+new BackupFileSystemManager(CONTINUOUS_BACKUP_REPLICATION_PEER, conf,
backupWalDir);
+ FileSystem fs = manager.getBackupFs();
+ Path walDir = manager.getWalsDir();
+ Path bulkloadDir = manager.getBulkLoadFilesDir();
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat(DATE_FORMAT);
+ dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
+
+ System.out.println("Listing directories under: " + walDir);
+
+ FileStatus[] directories = fs.listStatus(walDir);
+
+ for (FileStatus dirStatus : directories) {
+if (!dirStatus.isDirectory()) {
+ continue; // Skip files, we only want directories
+}
+
+Path dirPath = dirStatus.getPath();
+String dirName = dirPath.getName();
+
+try {
+ long dayStart = parseDayDirectory(dirName, dateFormat);
+ System.out
+.println("Checking WAL directory: " + dirName + " (Start Time: " +
dayStart + ")");
+
+ // If WAL files of that day are older than cutoff time, delete them
+ if (dayStart + ONE_DAY_IN_MILLISECONDS - 1 < cutoffTime) {
+System.out.println("Deleting outdated WAL directory: " + dirPath);
+fs.delete(dirPath, true);
Review Commen
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125701102
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
+BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
+ throws IOException {
+
+ Map backupTables =
sysTable.getContinuousBackupTableSet();
+ Set tablesToUpdate = new HashSet<>();
+
+ for (Map.Entry entry : backupTables.entrySet()) {
+if (entry.getValue() < cutoffTimestamp) {
+ tablesToUpdate.add(entry.getKey());
+}
+ }
+
+ if (!tablesToUpdate.isEmpty()) {
+sysTable.updateContinuousBackupTableSet(tablesToUpdate,
cutoffTimestamp);
+ }
+}
+
+/**
+ * Cleans up old WAL and bulk-loaded files based on the determined cutoff
timestamp.
+ */
+private void deleteOldWALFiles(Configuration conf, String backupWalDir,
long cutoffTime)
+ throws IOException {
+ System.out.println("Starting WAL cleanup in backup directory: " +
backupWalDir
++ " with cutoff time: " + cutoffTime);
+
+ BackupFileSystemManager manager =
+new BackupFileSystemManager(CONTINUOUS_BACKUP_REPLICATION_PEER, conf,
backupWalDir);
+ FileSystem fs = manager.getBackupFs();
+ Path walDir = manager.getWalsDir();
+ Path bulkloadDir = manager.getBulkLoadFilesDir();
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat(DATE_FORMAT);
+ dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
+
+ System.out.println("Listing directories under: " + walDir);
+
+ FileStatus[] directories = fs.listStatus(walDir);
+
+ for (FileStatus dirStatus : directories) {
+if (!dirStatus.isDirectory()) {
+ continue; // Skip files, we only want directories
+}
+
+Path dirPath = dirStatus.getPath();
+String dirName = dirPath.getName();
+
+try {
+ long dayStart = parseDayDirectory(dirName, dateFormat);
+ System.out
+.println("Checking WAL directory: " + dirName + " (Start Time: " +
dayStart + ")");
+
+ // If WAL files of that day are older than cutoff time, delete them
+ if (dayStart + ONE_DAY_IN_MILLISECONDS - 1 < cutoffTime) {
+System.out.println("Deleting outdated WAL directory: " + dirPath);
+fs.delete(dirPath, true);
Review Comment
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125640228
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
Review Comment:
True, we'll reuse that!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125625155
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
+BackupSystemTable sysTable = new BackupSystemTable(conn)) {
+// Get list of tables under continuous backup
+Map continuousBackupTables =
sysTable.getContinuousBackupTableSet();
+if (continuousBackupTables.isEmpty()) {
+ System.out.println("No continuous backups configured. Skipping WAL
cleanup.");
+ return;
+}
+
+// Find the earliest timestamp after which WALs are still needed
+long cutoffTimestamp = determineWALCleanupCutoffTime(sysTable);
+if (cutoffTimestamp == 0) {
+ System.err.println("ERROR: No valid full backup found. Skipping WAL
cleanup.");
+ return;
+}
+
+// Update metadata before actual cleanup to avoid inconsistencies
+updateBackupTableStartTimes(sysTable, cutoffTimestamp);
+
+// Delete WAL files older than cutoff timestamp
+deleteOldWALFiles(conf, backupWalDir, cutoffTimestamp);
+
+ }
+}
+
+/**
+ * Determines the cutoff time for cleaning WAL files.
+ * @param sysTable Backup system table
+ * @return cutoff timestamp or 0 if not found
+ */
+private long determineWALCleanupCutoffTime(BackupSystemTable sysTable)
throws IOException {
+ List backupInfos =
sysTable.getBackupInfos(BackupState.COMPLETE);
+ Collections.reverse(backupInfos); // Start from oldest
+
+ for (BackupInfo backupInfo : backupInfos) {
+if (BackupType.FULL.equals(backupInfo.getType())) {
+ return backupInfo.getStartTs();
+}
+ }
+ return 0;
+}
+
+/**
+ * Updates the start time for continuous backups if older than cutoff
timestamp.
+ * @param sysTableBackup system table
+ * @param cutoffTimestamp Timestamp before which WALs are no longer needed
+ */
+private void updateBackupTableStartTimes(BackupSystemTable sysTable, long
cutoffTimestamp)
+ throws IOException {
+
+ Map backupTables =
sysTable.getContinuousBackupTableSet();
+ Set tablesToUpdate = new HashSet<>();
+
+ for (Map.Entry entry : backupTables.entrySet()) {
+if (entry.getValue() < cutoffTimestamp) {
+ tablesToUpdate.add(entry.getKey());
+}
+ }
+
+ if (!tablesToUpdate.isEmpty()) {
+sysTable.updateContinuousBackupTableSet(tablesToUpdate,
cutoffTimestamp);
+ }
+}
+
+/**
+ * Cleans up old WAL and bulk-loaded files based on the determined cutoff
timestamp.
+ */
+private void deleteOldWALFiles(Configuration conf, String backupWalDir,
long cutoffTime)
+ throws IOException {
+ System.out.println("Starting WAL cleanup in backup directory: " +
backupWalDir
++ " with cutoff time: " + cutoffTime);
+
+ BackupFileSystemManager manager =
+new BackupFileSystemManager(CONTINUOUS_BACKUP_REPLICATION_PEER, conf,
backupWalDir);
+ FileSystem fs = manager.getBackupFs();
+ Path walDir = manager.getWalsDir();
+ Path bulkloadDir = manager.getBulkLoadFilesDir();
+
+ SimpleDateFormat dateFormat = new SimpleDateFormat(DATE_FORMAT);
+ dateFormat.setTimeZone(TimeZone.getTimeZone("UTC"));
+
+ System.out.println("Listing directories under: " + walDir);
+
+ FileStatus[] directories = fs.listStatus(walDir);
+
+ for (FileStatus dirStatus : directories) {
+if (!dirStatus.isDirectory()) {
+ continue; // Skip files, we only want directories
+}
+
+Path dirPath = dirStatus.getPath();
+String dirName = dirPath.getName();
+
+try {
+ long dayStart = parseDayDirectory(dirName, dateFormat);
+ System.out
+.println("Checking WAL directory: " + dirName + " (Start Time: " +
dayStart + ")");
+
+ // If WAL files of that day are older than cutoff time, delete them
+ if (dayStart + ONE_DAY_IN_MILLISECONDS - 1 < cutoffTime) {
+System.out.println("Deleting outdated WAL directory: " + dirPath);
+fs.delete(dirPath, true);
Review Commen
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125610012
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
Review Comment:
This connection creation is unnecessary I feel. Super class already has a
connection open. Please verify If you can reuse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r2125610012
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
Review Comment:
This connection creation is unnecessary I feel. Super class already has a
connection open. Please verify. If you can reuse it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
abhradeepkundu commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r212669
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +890,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (Strings.isNullOrEmpty(backupWalDir)) {
+System.out.println("No WAL directory specified for continuous backup.
Skipping cleanup.");
+return;
+ }
+
+ try (Connection conn = ConnectionFactory.createConnection(conf);
Review Comment:
NIT: avoid using generic name like conn, use specific like masterConn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
vinayakphegde commented on PR #7007: URL: https://github.com/apache/hbase/pull/7007#issuecomment-2936243821 > One more thing, do you think a lot of these `System.err/out.println()` statements can be replaced with `LOG.info/error()`? I know we want to give some feedback to the user via the Terminal, but it seems like a lot of these messages should go to the log (like the messages in `BackupCommands.updateBackupTableStartTimes()`, `BackupCommands. deleteOldWALFiles()`, etc.) Good point. we have lot of println lines everywhere in backup and restore code. let me create a new Jira to address this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2935652160
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 3m 37s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 5s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 27s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 20s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 6m 47s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 49s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 44s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 21m 55s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 49m 30s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux fea872204a93 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 331ca512ed6a9ab1e94e3429ce1f9d96d1cbef15 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/testReport/
|
| Max. process+thread count | 3604 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2935582891
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 33s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 1s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 28s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 49s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 15s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 44s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 1m 10s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 4m 20s | | the patch passed |
| +1 :green_heart: | compile | 0m 57s | | the patch passed |
| -0 :warning: | javac | 0m 57s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 4 new + 109 unchanged - 0 fixed = 113 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 20s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 1m 5s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 14m 30s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 56s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 14s | | The patch does not
generate ASF License warnings. |
| | | 38m 48s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux f964fe97bfd1 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / 331ca512ed6a9ab1e94e3429ce1f9d96d1cbef15 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/3/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
kgeisz commented on code in PR #7007:
URL: https://github.com/apache/hbase/pull/7007#discussion_r218391
##
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupCommands.java:
##
@@ -876,6 +889,140 @@ private boolean canAnyOtherBackupCover(List
allBackups, BackupInfo c
return false;
}
+/**
+ * Cleans up Write-Ahead Logs (WALs) that are no longer required for PITR
after a successful
+ * backup deletion.
+ */
+private void cleanUpUnusedBackupWALs() throws IOException {
+ Configuration conf = getConf() != null ? getConf() :
HBaseConfiguration.create();
+ String backupWalDir = conf.get(CONF_CONTINUOUS_BACKUP_WAL_DIR);
+
+ if (backupWalDir == null || backupWalDir.isEmpty()) {
Review Comment:
nit - You can use `Strings.isNullOrEmpty()` from
`org.apache.hbase.thirdparty.com.google.common.base`
```suggestion
if (Strings.isNullOrEmpty(backupWalDir)) {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2928976090
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 15s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 19s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 14s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 6m 1s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 6s | | the patch passed |
| +1 :green_heart: | compile | 0m 19s | | the patch passed |
| +1 :green_heart: | javac | 0m 19s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 13s | | the patch passed |
| +1 :green_heart: | shadedjars | 5m 58s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 22m 8s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 43m 8s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux 8d92f3b46603 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / bdc0de5c0f494fb52517faafb1896965a7a0c874 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/testReport/
|
| Max. process+thread count | 3317 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2928945045
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 32s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 16s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 31s | | HBASE-28957 passed |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-branch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-general-check/output/buildtool-branch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 31s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 0s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed |
| -0 :warning: | javac | 0m 29s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 4 new + 109 unchanged - 0 fixed = 113 total (was
109) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 9s |
[/buildtool-patch-checkstyle-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-general-check/output/buildtool-patch-checkstyle-hbase-backup.txt)
| The patch fails to run checkstyle in hbase-backup |
| +1 :green_heart: | spotbugs | 0m 35s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 7s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 44s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 30m 37s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux 6f8a4a4b4e6a 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / bdc0de5c0f494fb52517faafb1896965a7a0c874 |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 83 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/2/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2896962705
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 31s | | Docker mode activated. |
| -0 :warning: | yetus | 0m 3s | | Unprocessed flag(s):
--brief-report-file --spotbugs-strict-precheck --author-ignore-list
--blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck |
_ Prechecks _ |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 23s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 20s | | HBASE-28957 passed |
| +1 :green_heart: | javadoc | 0m 14s | | HBASE-28957 passed |
| +1 :green_heart: | shadedjars | 6m 5s | | branch has no errors when
building our shaded downstream artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 10s | | the patch passed |
| +1 :green_heart: | compile | 0m 20s | | the patch passed |
| +1 :green_heart: | javac | 0m 20s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 14s | | the patch passed |
| +1 :green_heart: | shadedjars | 6m 3s | | patch has no errors when
building our shaded downstream artifacts. |
_ Other Tests _ |
| -1 :x: | unit | 21m 38s |
[/patch-unit-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/artifact/yetus-jdk17-hadoop3-check/output/patch-unit-hbase-backup.txt)
| hbase-backup in the patch failed. |
| | | 43m 3s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | javac javadoc unit compile shadedjars |
| uname | Linux e21b98083ee4 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / c033241132754ceb4fb105d6aab677f3fb63148e |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Test Results |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/testReport/
|
| Max. process+thread count | 3714 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/console
|
| versions | git=2.34.1 maven=3.9.8 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [PR] HBASE-29255: Integrate backup WAL cleanup logic with the delete command [hbase]
Apache-HBase commented on PR #7007:
URL: https://github.com/apache/hbase/pull/7007#issuecomment-2896894275
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 30s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +0 :ok: | detsecrets | 0m 0s | | detect-secrets was not available.
|
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | hbaseanti | 0m 0s | | Patch does not have any
anti-patterns. |
_ HBASE-28957 Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 19s | | HBASE-28957 passed |
| +1 :green_heart: | compile | 0m 31s | | HBASE-28957 passed |
| +1 :green_heart: | checkstyle | 0m 10s | | HBASE-28957 passed |
| +1 :green_heart: | spotbugs | 0m 31s | | HBASE-28957 passed |
| +1 :green_heart: | spotless | 0m 46s | | branch has no errors when
running spotless:check. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 3m 4s | | the patch passed |
| +1 :green_heart: | compile | 0m 29s | | the patch passed |
| -0 :warning: | javac | 0m 29s |
[/results-compile-javac-hbase-backup.txt](https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/artifact/yetus-general-check/output/results-compile-javac-hbase-backup.txt)
| hbase-backup generated 4 new + 108 unchanged - 0 fixed = 112 total (was
108) |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 10s | | the patch passed |
| +1 :green_heart: | spotbugs | 0m 36s | | the patch passed |
| +1 :green_heart: | hadoopcheck | 12m 2s | | Patch does not cause any
errors with Hadoop 3.3.6 3.4.0. |
| +1 :green_heart: | spotless | 0m 45s | | patch has no errors when
running spotless:check. |
_ Other Tests _ |
| +1 :green_heart: | asflicense | 0m 10s | | The patch does not
generate ASF License warnings. |
| | | 30m 49s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.43 ServerAPI=1.43 base:
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/artifact/yetus-general-check/output/Dockerfile
|
| GITHUB PR | https://github.com/apache/hbase/pull/7007 |
| JIRA Issue | HBASE-29255 |
| Optional Tests | dupname asflicense javac spotbugs checkstyle codespell
detsecrets compile hadoopcheck hbaseanti spotless |
| uname | Linux fcf6d51a3ce9 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May
23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | HBASE-28957 / c033241132754ceb4fb105d6aab677f3fb63148e |
| Default Java | Eclipse Adoptium-17.0.11+9 |
| Max. process+thread count | 84 (vs. ulimit of 3) |
| modules | C: hbase-backup U: hbase-backup |
| Console output |
https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7007/1/console
|
| versions | git=2.34.1 maven=3.9.8 spotbugs=4.7.3 |
| Powered by | Apache Yetus 0.15.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
