[BackupPC-users] Retentiona and Archive
I have installed Backuppc succesfully and all is working well. I am having troubles understanding the retention policies for full and incremental. I am looking at being able to restore one particular file from any day in the past year. I have read about archiving off, but not sure how to set the retention time on this either. Some help or explanation would be great. I have read the docs, and dont completely understand it. Many thanks Max -- Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Retentiona and Archive
Hi Max,
can you be more precise on the part in the docs you don’t understand that well?
If your want to be able to have more than 365 days of retention I would suggest
this as a possible configuration:
$Conf{FullAgeMax} = 373;
$Conf{FullKeepCntMin} = 53;
$Conf{FullPeriod} = 6.97;
You should adjust your Incremental-Parameters as well. Since the full period is
approximately 7, multiplied with 53 as a FullKeepCntMin you should have 371
days or more than one year of retention.
For this specific part the documentation is pretty nice since it offers
examples for that:
http://backuppc.sourceforge.net/faq/BackupPC.html#_conf_fullkeepcnt_
> Am 17.01.2016 um 14:42 schrieb Max Ricketts :
>
> I have installed Backuppc succesfully and all is working well. I am having
> troubles understanding the retention policies for full and incremental.
>
> I am looking at being able to restore one particular file from any day in the
> past year.
>
> I have read about archiving off, but not sure how to set the retention time
> on this either.
>
> Some help or explanation would be great. I have read the docs, and dont
> completely understand it.
>
> Many thanks
>
> Max
> --
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
--
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BPC 4 very slow
On 17/01/16 05:53, Alexander Moisseev wrote: > On 16.01.16 15:04, Gandalf Corvotempesta wrote: >> I can't use v3 because i'm having servers with millions of file to be >> transfered and v3 takes ages in "Building files list". > You mentioned earlier you are trying to backup dovecot with Maildir. Do you > have considered to move from Maildir to mdbox? It will definitely help with > BackupPC and rsync performance. More over you will be able to use dovecot > tools (+ your own scripts) or dovecot replication feature instead of > rsync/BackupPC. It will increase performance dramatically. I'm not sure of the exact numbers you are referring to (millions is 1 million or 50 million?, also "ages" is 5 minutes or 5 hours). In any case, back in 2012, I did a full backup of one of my mail servers, and it was completed in 70 minutes, with approx 1.3 million files (that would have been with BPC v3). Equally, a more recent backup with v4 shows it took 402 minutes (almost 7 hours) and the quickest (recent BPC v4) full backup took 146 minutes. So, while generating the file list might take a long time with BPC v3, I don't think it should significantly change the overall backup time. I would expect my BPC server has gotten quicker over the years, as has the mail server. Variations are likely due to concurrent backups, and bandwidth limitations. It would be interesting to hear an update of where you are at, and what has happened (either good or bad). Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BPC 4 very slow
On 16/01/16 23:04, Gandalf Corvotempesta wrote: > 2016-01-14 23:00 GMT+01:00 Adam Goryachev > : >> I would suggest anyone considering BPC should use v3 unless they >> specifically need the v4 features, and even then, are prepared for the >> possible shortcomings inherent in using alpha software. > I can't use v3 because i'm having servers with millions of file to be > transfered and v3 takes ages in "Building files list". > v4 uses native rsync and incremental file list. This is faster, > transfer start immediatly, not after many many hours. > > BTW, 1 server was backuped properly, now is running "fsck #0". As I > understood from docs, BackupPC should not run when > fsck in running. I'm having a full dump of srv1, and srv2 that is > doing "fsck #0". Is this bad or fsck can be run parallel with other > backups? AFAIK, when you see fsck #0 on the web interface, it is actually doing a refCountUpdate, which just updates the current host, therefore it doesn't matter if other hosts are running a concurrent backup. > Usually how long does it take to complete? this is slowing down the > other backup that seems to be frozen. This is dependant on the number of files in the backup, and especially, your disk I/O performance. You will probably want to think about RAID10 or similar for your backup server in order to massively improve performance (or use bigger caches, either hardware raid cache, or allow linux to cache more). Equally, your filesystem might need tuning, or changing to a different FS might help too. Running multiple concurrent backups might be slower than running one at a time, depending on the speed of the client, and the performance of your BPC server. Remember, the disk can only read or write from one sector at a time, and random performance is significantly worse. Regards, Adam -- Adam Goryachev Website Managers www.websitemanagers.com.au -- Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BPC 4 very slow
On 16.01.16 22:16, Gandalf Corvotempesta wrote: > 2016-01-16 19:53 GMT+01:00 Alexander Moisseev : >> You mentioned earlier you are trying to backup dovecot with Maildir. Do you >> have considered to move from Maildir to mdbox? It will definitely help with >> BackupPC and rsync performance. More over you will be able to use dovecot >> tools (+ your own scripts) or dovecot replication feature instead of >> rsync/BackupPC. It will increase performance dramatically. > > mdbox is available since version 2. Some server are very old, with dovecot 1. > I don't think old hardware will be a problem for version 2. I am using low end hardware for a small mail server (P4 2.80GHz, 2GB RAM, gmirror of 2 SATA drives) and BackupPC server (P4 2.80GHz, 1.5GB RAM, SATA drive) and 100 Mbps networking. About 2 years ago I was using Maildir and BPC v3 (rsyncd with checksum caching). Mail storage had about 100 thousands files. The first two full backups was completed in about 4 hours each. Subsequent full backups was taken about 30 minutes (if there is no concurrent backups). I never tried BPC v4 to backup Maildir. Then I converted mail storage to mdbox + SIS (single instance attachment storage) + compression. I never tried BPC with mdbox, I see no point, but I believe file list building should be much quicker since file number is significantly lower. I installed dovecot (mdbox + SIS + compression) on a test machine with similar hardware configuration. Initial backup with "doveadm backup" took about 1-3 hours, I don't remember exact time. Subsequent everyday backups take about 1-2 minutes and don't depend on mail storage size or messages number, but on changes made between backups. Finally for the backup end I've installed dovecot (mdbox + SIS + compression) on a server with ZFS. Now, in order to have retrospective, I'm taking snapshot after each backup and using BackupPC-like aging. And "zfs send" to make offline copies. Hostname| Level | Size (MB) | Written (MB) | Used (MB) | Last available | Days old tank1/vmail | 0 | 27860.45 | 23650.66 | 2128.86 | 2015-07-11 07:51 | 191 tank1/vmail | 0 | 27804.61 | 2073.86 |612.73 | 2015-08-12 07:51 | 159 tank1/vmail | 0 | 29287.60 | 2227.53 |686.91 | 2015-09-13 07:51 | 127 tank1/vmail | 0 | 30896.84 | 2488.86 |601.22 | 2015-10-15 07:51 | 95 tank1/vmail | 0 | 35778.12 | 6177.88 | 1852.94 | 2015-11-16 07:51 | 63 tank1/vmail | 0 | 37257.94 | 4151.30 |443.62 | 2015-12-18 07:51 | 31 tank1/vmail | 0 | 37257.47 | 519.09 |186.52 | 2015-12-22 07:51 | 27 tank1/vmail | 0 | 37273.77 | 545.84 |191.96 | 2015-12-26 07:51 | 23 tank1/vmail | 0 | 37564.53 | 686.51 |165.02 | 2015-12-30 07:51 | 19 tank1/vmail | 0 | 37533.15 | 415.20 |129.96 | 2016-01-03 07:51 | 15 tank1/vmail | 0 | 37454.53 | 220.35 | 99.68 | 2016-01-07 07:51 | 11 tank1/vmail | 0 | 37419.07 | 170.54 |107.71 | 2016-01-11 07:51 | 7 tank1/vmail | 0 | 37520.83 | 317.79 |112.94 | 2016-01-12 07:51 | 6 tank1/vmail | 0 | 37633.12 | 328.19 |113.92 | 2016-01-13 07:51 | 5 tank1/vmail | 0 | 37792.96 | 345.29 | 83.54 | 2016-01-14 07:51 | 4 tank1/vmail | 0 | 37961.14 | 302.97 | 61.54 | 2016-01-15 07:51 | 3 tank1/vmail | 0 | 38045.83 | 178.00 | 12.08 | 2016-01-16 07:51 | 2 tank1/vmail | 0 | 38060.19 |31.03 | 15.53 | 2016-01-17 07:51 | 1 tank1/vmail | 0 | 38029.69 |38.44 | 0.00 | 2016-01-18 07:51 | 0 tank1/vmail | - | 38029.69 | 0.00 | 32687.01 |- | - NAME PROPERTY VALUE tank1/vmail used 43.8G tank1/vmail logicalused 51.1G Actual total size of mail stored on the mailserver (prior SIS and compression) 62GB, 170 thousands of messages. I know it is a very small server, but you can extrapolate the results. -- Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BPC 4 very slow
On 18.01.16 4:46, Adam Goryachev wrote: > So, while generating the file list might take a long time with BPC v3, I > don't think it should significantly change the overall backup time. Assuming checksum caching configured, for first and second backup, yes. But it is not true for subsequent backups since most of Maildir files never change over time. -- Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
