[jira] [Comment Edited] (HBASE-16458) Shorten backup / restore test execution time
[ https://issues.apache.org/jira/browse/HBASE-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608470#comment-16608470 ] Ted Yu edited comment on HBASE-16458 at 9/9/18 4:29 PM: HBASE-16458-v2.patch was the last one from Vlad. Patch v5 is from me, based on Vlad's v2. was (Author: yuzhih...@gmail.com): HBASE-16458-v2.patch was the last one from Vlad. Patch v5 is from me. > Shorten backup / restore test execution time > > > Key: HBASE-16458 > URL: https://issues.apache.org/jira/browse/HBASE-16458 > Project: HBase > Issue Type: Test >Reporter: Ted Yu >Assignee: Vladimir Rodionov >Priority: Major > Labels: backup > Attachments: 16458-v1.patch, 16458.HBASE-7912.v3.txt, > 16458.HBASE-7912.v4.txt, 16458.v1.txt, 16458.v2.txt, 16458.v2.txt, > 16458.v3.txt, 16458.v4.txt, 16458.v5.txt, HBASE-16458-v1.patch, > HBASE-16458-v2.patch > > > Below was timing information for all the backup / restore tests (today's > result): > {code} > Running org.apache.hadoop.hbase.backup.TestIncrementalBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 576.273 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackup > Running org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 124.67 sec - > in org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Running org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.34 sec - > in org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Running org.apache.hadoop.hbase.backup.TestBackupAdmin > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 490.251 sec - > in org.apache.hadoop.hbase.backup.TestBackupAdmin > Running org.apache.hadoop.hbase.backup.TestHFileArchiving > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.323 sec - > in org.apache.hadoop.hbase.backup.TestHFileArchiving > Running org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.492 sec - > in org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Running org.apache.hadoop.hbase.backup.TestBackupDescribe > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.758 sec - > in org.apache.hadoop.hbase.backup.TestBackupDescribe > Running org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.187 sec - > in org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Running org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 330.539 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Running org.apache.hadoop.hbase.backup.TestRemoteBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.371 sec - > in org.apache.hadoop.hbase.backup.TestRemoteBackup > Running org.apache.hadoop.hbase.backup.TestBackupSystemTable > Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.893 sec - > in org.apache.hadoop.hbase.backup.TestBackupSystemTable > Running org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.779 sec - > in org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Running org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 117.815 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Running org.apache.hadoop.hbase.backup.TestBackupShowHistory > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 136.517 sec - > in org.apache.hadoop.hbase.backup.TestBackupShowHistory > Running org.apache.hadoop.hbase.backup.TestRemoteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 91.799 sec - > in org.apache.hadoop.hbase.backup.TestRemoteRestore > Running org.apache.hadoop.hbase.backup.TestFullRestore > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 317.711 sec > - in org.apache.hadoop.hbase.backup.TestFullRestore > Running org.apache.hadoop.hbase.backup.TestFullBackupSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.045 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSet > Running org.apache.hadoop.hbase.backup.TestBackupDelete > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.214 sec - > in org.apache.hadoop.hbase.backup.TestBackupDelete > Running org.apache.hadoop.hbase.backup.TestBackupDeleteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.631 sec - > in
[jira] [Comment Edited] (HBASE-16458) Shorten backup / restore test execution time
[ https://issues.apache.org/jira/browse/HBASE-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434053#comment-15434053 ] Ted Yu edited comment on HBASE-16458 at 8/24/16 1:57 AM: - I forgot to record the exact runtime without patch - on Linux. There were several minutes of savings. was (Author: yuzhih...@gmail.com): I forgot to record the runtime without patch - on Linux. There were several minutes of savings. > Shorten backup / restore test execution time > > > Key: HBASE-16458 > URL: https://issues.apache.org/jira/browse/HBASE-16458 > Project: HBase > Issue Type: Test >Reporter: Ted Yu > Labels: backup > Attachments: 16458.v1.txt, 16458.v2.txt, 16458.v3.txt > > > Below was timing information for all the backup / restore tests (today's > result): > {code} > Running org.apache.hadoop.hbase.backup.TestIncrementalBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 576.273 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackup > Running org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 124.67 sec - > in org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Running org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.34 sec - > in org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Running org.apache.hadoop.hbase.backup.TestBackupAdmin > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 490.251 sec - > in org.apache.hadoop.hbase.backup.TestBackupAdmin > Running org.apache.hadoop.hbase.backup.TestHFileArchiving > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.323 sec - > in org.apache.hadoop.hbase.backup.TestHFileArchiving > Running org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.492 sec - > in org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Running org.apache.hadoop.hbase.backup.TestBackupDescribe > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.758 sec - > in org.apache.hadoop.hbase.backup.TestBackupDescribe > Running org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.187 sec - > in org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Running org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 330.539 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Running org.apache.hadoop.hbase.backup.TestRemoteBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.371 sec - > in org.apache.hadoop.hbase.backup.TestRemoteBackup > Running org.apache.hadoop.hbase.backup.TestBackupSystemTable > Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.893 sec - > in org.apache.hadoop.hbase.backup.TestBackupSystemTable > Running org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.779 sec - > in org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Running org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 117.815 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Running org.apache.hadoop.hbase.backup.TestBackupShowHistory > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 136.517 sec - > in org.apache.hadoop.hbase.backup.TestBackupShowHistory > Running org.apache.hadoop.hbase.backup.TestRemoteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 91.799 sec - > in org.apache.hadoop.hbase.backup.TestRemoteRestore > Running org.apache.hadoop.hbase.backup.TestFullRestore > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 317.711 sec > - in org.apache.hadoop.hbase.backup.TestFullRestore > Running org.apache.hadoop.hbase.backup.TestFullBackupSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.045 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSet > Running org.apache.hadoop.hbase.backup.TestBackupDelete > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.214 sec - > in org.apache.hadoop.hbase.backup.TestBackupDelete > Running org.apache.hadoop.hbase.backup.TestBackupDeleteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.631 sec - > in org.apache.hadoop.hbase.backup.TestBackupDeleteRestore > Running org.apache.hadoop.hbase.backup.TestIncrementalBackupDeleteTable > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 190.358 sec - > in
[jira] [Comment Edited] (HBASE-16458) Shorten backup / restore test execution time
[ https://issues.apache.org/jira/browse/HBASE-16458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15434048#comment-15434048 ] Vladimir Rodionov edited comment on HBASE-16458 at 8/24/16 1:50 AM: Without patch - 34 minutes With patch - 29 minutes On my laptop [~tedyu], do you have improvement numbers? Otherwise, looks good. was (Author: vrodionov): Without patch - 34 minutes With patch - 29 minutes On my laptop [~tedyu], do you have improvement numbers? > Shorten backup / restore test execution time > > > Key: HBASE-16458 > URL: https://issues.apache.org/jira/browse/HBASE-16458 > Project: HBase > Issue Type: Test >Reporter: Ted Yu > Labels: backup > Attachments: 16458.v1.txt, 16458.v2.txt, 16458.v3.txt > > > Below was timing information for all the backup / restore tests (today's > result): > {code} > Running org.apache.hadoop.hbase.backup.TestIncrementalBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 576.273 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackup > Running org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 124.67 sec - > in org.apache.hadoop.hbase.backup.TestBackupBoundaryTests > Running org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 102.34 sec - > in org.apache.hadoop.hbase.backup.TestBackupStatusProgress > Running org.apache.hadoop.hbase.backup.TestBackupAdmin > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 490.251 sec - > in org.apache.hadoop.hbase.backup.TestBackupAdmin > Running org.apache.hadoop.hbase.backup.TestHFileArchiving > Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.323 sec - > in org.apache.hadoop.hbase.backup.TestHFileArchiving > Running org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.492 sec - > in org.apache.hadoop.hbase.backup.TestSystemTableSnapshot > Running org.apache.hadoop.hbase.backup.TestBackupDescribe > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.758 sec - > in org.apache.hadoop.hbase.backup.TestBackupDescribe > Running org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 109.187 sec - > in org.apache.hadoop.hbase.backup.TestBackupLogCleaner > Running org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 330.539 sec - > in org.apache.hadoop.hbase.backup.TestIncrementalBackupNoDataLoss > Running org.apache.hadoop.hbase.backup.TestRemoteBackup > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.371 sec - > in org.apache.hadoop.hbase.backup.TestRemoteBackup > Running org.apache.hadoop.hbase.backup.TestBackupSystemTable > Tests run: 15, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 67.893 sec - > in org.apache.hadoop.hbase.backup.TestBackupSystemTable > Running org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.779 sec - > in org.apache.hadoop.hbase.backup.TestRestoreBoundaryTests > Running org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 117.815 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSetRestoreSet > Running org.apache.hadoop.hbase.backup.TestBackupShowHistory > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 136.517 sec - > in org.apache.hadoop.hbase.backup.TestBackupShowHistory > Running org.apache.hadoop.hbase.backup.TestRemoteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 91.799 sec - > in org.apache.hadoop.hbase.backup.TestRemoteRestore > Running org.apache.hadoop.hbase.backup.TestFullRestore > Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 317.711 sec > - in org.apache.hadoop.hbase.backup.TestFullRestore > Running org.apache.hadoop.hbase.backup.TestFullBackupSet > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 87.045 sec - > in org.apache.hadoop.hbase.backup.TestFullBackupSet > Running org.apache.hadoop.hbase.backup.TestBackupDelete > Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 86.214 sec - > in org.apache.hadoop.hbase.backup.TestBackupDelete > Running org.apache.hadoop.hbase.backup.TestBackupDeleteRestore > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.631 sec - > in org.apache.hadoop.hbase.backup.TestBackupDeleteRestore > Running org.apache.hadoop.hbase.backup.TestIncrementalBackupDeleteTable > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0,