On 2018-12-12 19:11, Debra S Baddorf wrote:
Oh, that’s right — Chris DID tell us what he was trying to do, last week: (in case it helps with any further answers)
OK, given that what you want appears to be frequent snapshots, Amanda is almost certainly _not_ the correct tool for this job for three reasons:
* It can not do atomic snapshots of the filesystem state, so either you need to freeze all write I/O to the DLE, or you won't have coherent copy of the state of the DLE from when you ran the backup. This doesn't matter most of the time for regular backup usage, because you just run Amanda during off-hours when nobody's doing anything and the system is idle, but for this it's going to be a potential issue.
* It takes a _lot_ of system resources to do a backup with Amanda. This is mitigated by your proposed approach of doing constantly increasing incremental levels, but even for that Amanda has to call `stat()` on _everything_ in the DLE.
* It's not trivial (as you have found out) to get this type of thing to work reliably.
I would suggest looking at the following alternative approaches:* Use ZFS, BTRFS, or another filesystem that supports native snapshotting, and just do snapshots regularly. This is likely to be your best approach. In some cases, depending on the platform and filesystem, you may not even need to do anything (for example, NILFS2 on Linux has implicit snapshots built in because it's a log structured filesystem).
* Store all the data on a NAS device that can do snapshots (for example, something running FreeNAS), and have it do regular snapshots. This largely reduces to the above, just indirected over the network.
* Use a filesystem that supports automatic native file versioning. The classic example is Files-11 from OpenVMS. Other options for this include GitFS [1], CopyFS [2], Plan 9's Fossil filesystem
* Store all the data on a NAS device that does automatic native file versioning.
* If all else fails, you can technically do this with Amanda by using `amadmin force` to force the level 0 dump, and `amadmin force-bump` for each backup _after the second_ (the first backup after a level 0 will always be a level 1) to get the increasing incrementals.
[1] https://www.presslabs.com/code/gitfs/ [2] https://boklm.eu/copyfs/
On Dec 7, 2018, at 12:04 PM, Chris Miller <c...@tryx.org> wrote: Hi Folks, I'm about to start a project during which I want to be able to: • Request a backup at any moment and have that backup be either an incremental backup (Level N+1), meaning everything that has changed since the last backup (Level N), or a differential backup, meaning everything that has changed since the last full backup (Level 1). The second provision, "differential backup" is pretty straight forward, but I have no idea how to configure a constantly increasing dump level. • The first backup of the day, meaning the first backup after midnight, will be a full filesystem backup. Discussion on point 1: The provision is for capturing changes that occur during a given period of time, and no so much for "backup" per se, so AMANDA may not be the best tool, but it is what I have, so I'm tying to make it fit. I know how to request a backup, so that's not my problem, but I don't know how to force a given level. In particular, I don't know how to force a Level N+1 backup. I could replace the Level N+1 requirement with a forced Level 1, run my experiment, and force a level 2, and this would meet my requirement of capturing all the changes during a particular interval. But, again, that requires forcing AMANDA to take direction about backup levels and I don't know how to do that. Before anybody reminds me that this is why god invented git, I would like to add, that the scope of git is typically only known parts of the project, and I want to capture log files and other files that are sometimes created in temporary locations with temporary names, which are not known a priori and therefore can't be "managed" with git. Discussion on point 2: The "first backup of the day", will run as a cron job, but it must be a level 0, full filesystem backup so no work for the day is lost. It is more forcing AMANDA to take direction, I don't know exactly how to do this. I don't think I like the idea of forcing AMANDA (#MeToo) to do things, but I'm not above payment in kind. (-: Thanks for the help,
