On 2017-03-04 13:12, Craig Barratt wrote:
> BackupPC community,
>
> I'm happy to announce that BackupPC 4.0.0 has been released on Github [1] and
> SourceForge [2].
>
> BackupPC 4.0.0 is a significant improvement over 3.x in terms of performance
> and storage efficiency. It is backward compatible with 3.x storage, so it can
> be used to upgrade an existing V3 installation as well as for brand new
> installs.
>
> BackupPC 4.0.0 requires the perl module BackupPC::XS [3] (>= 0.50) and
> rsync-bpc [4] (>= 3.0.9.5).
>
> After installing those two packages, BackupPC 4.0.0 can be installed from the
> tar ball with:
>
> tar zxf BackupPC-4.0.0.tar.gz
> cd BackupPC-4.0.0
> perl configure.pl [5]
>
> See the README.md, ChangeLog and doc/BackupPC.html [6] files for more
> information. An overview of the major changes is below.
>
> Thanks for to everyone who provided fixes and reported issues with the alpha
> and github versions.
>
> Enjoy!!
>
> Craig
>
> Here's a summary of the most significant changes in V4:
>
> * No use of hardlinks (except temporarily to do atomic renames). Reference
> counting is handled at the application level in a batch manner (hardlinks
> will still remain for any legacy V3 backups).
> * Backups are stored as "reverse deltas" - the most recent backup is always
> filled and older backups are reconstituted by merging all the deltas starting
> with the nearest future filled backup and working backwards.
> * This is the opposite of V3 where incrementals are stored as "forward
> deltas" to a prior backup (typically the last full backup or prior
> lower-level incremental backup, or the last full in the case of rsync).
> * Since the most recent backup is filled, viewing/restoring that backup
> (which is the most common backup used) doesn't require merging any deltas
> from other backups.
> * The concepts of incr/full backups and unfilled/filled storage are
> decoupled. The most recent backup is always filled. By default, for the
> remaining backups, full backups are filled and incremental backups are
> unfilled, but that is configurable.
> * Uses full-file MD5 digests, which are stored in the directory attrib files.
> Each backup directory only contains an empty attrib file whose name includes
> its own MD5 digest, which is used to look up the attrib file's contents in
> the pool. In turn, that file contains the metadata for every file in that
> directory, including each files's MD5 digest.
> * The Pool layout still supports chains to handle md5 collisions. While
> collisions can be constructed and are now well-known, they are highly
> unlikely in the wild. Pool files are never renamed or moved, unlike V3.
> * Any backup can be deleted (deltas are merged into next older backup if it
> is not filled).
> * The reverse deltas allow "infinite incrementals" - no need for a full
> backup if you are willing to trade speed for the risk that a file change will
> not be detected if the metadata (eg, mtime or size) doesn't change.
> * An rsync "full" backup now uses --checksum (instead of --ignore-times),
> which is much more efficient on the server side - the server just needs to
> check the full-file checksum computed by the client, together with the mtime,
> nlinks, size attributes, to see if the file has changed. If you want a more
> conservative approach, you can change it back to --ignore-times, which
> requires the server to send block checksums to the client.
> * The use of rsync --checksum allows BackupPC to guess a potential match
> anywhere in the pool, even on a first-time backup. In that case, the usual
> rsync block checksums are still exchanged to make sure the complete file is
> identical.
> * Uses a modified rsync called rsync_bpc (currently based on rsync-3.0.9) on
> the server side (in place of File::RsyncP), with a C code interface to the
> BackupPC storage. So the whole data path for rsync is now in compiled C code,
> which is much faster than perl.
> * Due to the use of rsync-3.X, acls and xattrs are supported, and many other
> useful options (but not all) are supported. Rsync protocol 30 supports the
> efficient incremental file list, which significantly improves memory usage
> and startup time. It also supports MD5 full-file checksums, which match
> BackupPC's new digest. That allows a full-file digest to be checked as easily
> as an mtime on the server side.
> * Significant portions of the BackupPC code are now compiled C code in a new
> module called BackupPC::XS that is dynamically linked to perl.
>
> * V3 migration: nothing specific is needed. V4 can browse/view/restore V3
> backups. When you install V4, no changes are made to any V3 backups. If you
> are upgrading from V3, be sure to set $Conf{PoolV3Enabled} to 1 so the old V3
> pool is searched for matching files.
>
> * When you install V4, it will notice that the V3 pool exists. Running
> configure.pl [5] should set $Conf{PoolV3Enabled} to 1 in that case, but you
> should be sure to check that.
> * When a V4 backup is first done, BackupPC_backupDuplicate is run to
> duplicate the most recent V3 backup to create a new V4 backup. A "filled"
> view of the most recent V3 backup is used to create a "filled" V4 backup tree.
> * This step could be time consuming, since every file needs to be read (as a
> V3 file) and written as a V4 file. However, the V4 pooling code knows about
> the V3 pool, so it will move the V3 pool file into the V4 pool. So this
> duplication process doesn't burn a lot of pool storage space, but every file
> still needs to be read (to compute the MD5 digest) and "written" (really just
> matching/linking).
> * Expiry: all the V3 + V4 backups are considered on a combined basis for
> expiry checking.
> * On a clean new V4 install, the steps of computing and checking V3 digests
> is eliminated.
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
Congratulations on this significant milestone.
--
_______________________________
Bob Wooden of Donelson Trophy
Links:
------
[1] https://github.com/backuppc/backuppc/releases
[2] https://sourceforge.net/projects/backuppc/files/backuppc/4.0.0/
[3] https://github.com/backuppc/backuppc-xs/releases
[4] https://github.com/backuppc/rsync-bpc/releases
[5] http://configure.pl
[6] http://backuppc.sourceforge.net/BackupPC-4.0.0.html
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki: http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
