And if you think application consistency is nothing to worry about know that Git on Ext4 has failed to recover.
My current conclusion: 1. If I want application consistency I have to take down any application that has an internal database or backup those applications prior to FS backup. 2. rsnapshot <http://rsnapshot.org/>is a good candidate. It matches the way I want to use backup and it is abased on rsync that I am familiar with I'd like to consider using CCFS, but I have no instructions for that. Linux FS snapshot seems to be of a temporary nature, because it will grow over time (it is a diff between snapshot time and current time). LVM is a learning curve I have managed to escape so far: The generic way to make snapshots under Linux is at the level of the > storage volume. Your filesystem must be on an LVM logical volume > <https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)>, which is > Linux's own partition system, as opposed to directly on a platform-native > disk partition. So, my existing installations probably avoids LVM and snapshot is not an option. On Mon, May 8, 2017 at 3:46 PM, Sergei G <sergeig.pub...@gmail.com> wrote: > We have to remember that things are actually worse at the application > consistency level. An application may think that it has committed its > write, but the file system has not written all the bytes to the disk yet. > > File system crash consistency is a sibling of the backup topic. Here is > relevant article I run into a some time ago https://blog.acolyer.org/2017/ > 03/15/application-crash-consistency-and-performance-with-ccfs/ > > According to link above Ext4 FS suffers from these inconsistencies. > > On Sun, May 7, 2017 at 7:56 AM, <to...@tuxteam.de> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Sun, May 07, 2017 at 08:42:36AM -0400, rhkra...@gmail.com wrote: >> > On Sunday, May 07, 2017 05:47:20 AM to...@tuxteam.de wrote: >> > >> > <good stuff elided> >> > >> > > Otherwise, "on line" backup is simply not an option. >> >> [...] >> >> > I guess the way that comes to mind, maybe somewhat implied by what >> you've >> > written (or at least inferred by me ;-) would be to take down one or >> more of >> > the servers at a time, make static backups of those, then restore those >> to >> > service and repeat until all have been backed up, very likely repeating >> that >> > continuously. >> >> "Taking down" as in "stopping all relevant applications" (shutting down >> the whole thing would be a superset of that, of course ;) >> >> [...] >> >> > Oh, ok, I guess (well, I know RAID isn't really for backup, it's for >> uptime as >> > is often stated on this and other lists), but something like RAID where >> > relevant data is written to more than one disk and / or storage "farm", >> one >> > being the "in service" device (to serve data to the app in "real time") >> and >> > one or more just recording data as backup. >> >> This is a variant on the snapshot pattern. You take a snapshot of the >> file system and back up that. You are guaranteed that the *file system* >> backup is consistent (well, modulo bugs and glitches), but since the >> applications don't have a clue what's going on, application state >> might still be inconsistent, unless they go out of their way to keep >> a consistent state on-disk all of the time (in the above scenario >> of shutdown, we assumed that the application leaves a consistent >> state after being shut down, which is a reasonable assumption, I'd >> say). >> >> That said, and as you can see in the PostgreSQL example, with a little >> help of your application, the snapshot thing works (in the case of >> PostgreSQL things are much more relaxed and you can usually go with >> plain rsync, for example, without resorting to snapshot). >> >> All in all I'm pretty happy with a plain, straight rsync-style backup >> for my workstation. The probability of a busted backup is so low that >> I fear more for my backup medium going sour. >> >> On a high-churn machine with very valuable data things might look >> differently, though... >> >> regards >> - -- tomás >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG v1.4.12 (GNU/Linux) >> >> iEYEARECAAYFAlkPNaIACgkQBcgs9XrR2kYGWgCfcR14ZU9wA9kKDampxUY7MCHb >> IjUAnj0zjx2U5yFy7TIIO/rP5UO7E6QI >> =bD6R >> -----END PGP SIGNATURE----- >> >> >
