Re: Live Fille System Backup

2017-05-24 Thread Sergei G
thank you for mentioning bup. That's a great option I am considering. On Fri, May 12, 2017 at 5:52 AM, Stefan Monnier wrote: > > I would like a backup tool that does not bring a million dependencies > with > > MBs of files. Something that works on server without X Windows and can > > send back

Re: Live Fille System Backup

2017-05-12 Thread Stefan Monnier
> I would like a backup tool that does not bring a million dependencies with > MBs of files. Something that works on server without X Windows and can > send backup to an externally attached USB drive. Nothing fancy. No > network infrastructure. Incremental backups would be greatly appreciated.

Re: Live Fille System Backup

2017-05-09 Thread Curt
On 2017-05-09, Greg Wooledge wrote: > On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: >> Because someone mentioned Btrfs and its backup "magic," I'm looking at >> the wiki, which says (amongst other things): >> >> https://btrfs.wiki.kernel.org/index.php/Incremental_Backup >> >> Instant, A

Re: Live Fille System Backup

2017-05-09 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 09, 2017 at 08:08:48AM -0400, Greg Wooledge wrote: [...] > So, basically the same as LVM snapshots, which are independent of the > type of file system that sits on the LVM logical volume. To be fair, btrfs knows about the file system, wh

Re: Live Fille System Backup

2017-05-09 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: [...] > Because someone mentioned Btrfs and its backup "magic," I'm looking at > the wiki, which says (amongst other things): > > https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > >

Re: Live Fille System Backup

2017-05-09 Thread Greg Wooledge
On Tue, May 09, 2017 at 12:03:14PM +, Curt wrote: > Because someone mentioned Btrfs and its backup "magic," I'm looking at > the wiki, which says (amongst other things): > > https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > > Instant, Atomic COW Snapshots > Since the snapshots a

Re: Live Fille System Backup

2017-05-09 Thread Curt
On 2017-05-09, to...@tuxteam.de wrote: > > On Mon, May 08, 2017 at 04:03:54PM -0700, Sergei G wrote: >> And if you think application consistency is nothing to worry about know >> that Git on Ext4 has failed to recover. >> >> My current conclusion: >> >> 1. If I want application consistency I hav

Re: Live Fille System Backup

2017-05-09 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Mon, May 08, 2017 at 04:03:54PM -0700, Sergei G wrote: > And if you think application consistency is nothing to worry about know > that Git on Ext4 has failed to recover. > > My current conclusion: > > 1. If I want application consistency I have t

Re: Live Fille System Backup

2017-05-08 Thread Sergei G
And if you think application consistency is nothing to worry about know that Git on Ext4 has failed to recover. My current conclusion: 1. If I want application consistency I have to take down any application that has an internal database or backup those applications prior to FS backup. 2. rsnapsh

Re: Live Fille System Backup

2017-05-08 Thread Sergei G
We have to remember that things are actually worse at the application consistency level. An application may think that it has committed its write, but the file system has not written all the bytes to the disk yet. File system crash consistency is a sibling of the backup topic. Here is relevant a

Re: Live Fille System Backup

2017-05-07 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, May 07, 2017 at 08:42:36AM -0400, rhkra...@gmail.com wrote: > On Sunday, May 07, 2017 05:47:20 AM to...@tuxteam.de wrote: > > > > > Otherwise, "on line" backup is simply not an option. [...] > I guess the way that comes to mind, maybe some

Re: Live Fille System Backup

2017-05-07 Thread rhkramer
On Sunday, May 07, 2017 05:47:20 AM to...@tuxteam.de wrote: > Otherwise, "on line" backup is simply not an option. Thanks! I've been paying at least peripheral attention to this thread (as I do to many), and got to thinking / wondering about how to deal with backup in an "enterprise" type of

Re: Live Fille System Backup

2017-05-07 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sun, May 07, 2017 at 10:53:51AM +1200, Ben Caradoc-Davies wrote: [also a reply to Henrique, elsewhere in this thread] > If a file is updated while it is being copied, it may contain only > half a change set and be in an internally inconsistent sta

Re: Live Fille System Backup

2017-05-06 Thread Ben Caradoc-Davies
On 07/05/17 01:56, to...@tuxteam.de wrote: On Sat, May 06, 2017 at 06:42:46AM -0400, Dan Ritter wrote: On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: it would be naive at best to think that busy files [...] What is a

Re: Live Fille System Backup

2017-05-06 Thread Henrique de Moraes Holschuh
On Sat, 06 May 2017, to...@tuxteam.de wrote: > > 2. one or more components of a set of several files change *during* > >backup (files may be internally consistent, but the set of files > >itself isn't). > > Only the application can know that. But this is a realm where snapshots > won't hel

Re: Live Fille System Backup

2017-05-06 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, May 06, 2017 at 02:18:47PM -0300, Henrique de Moraes Holschuh wrote: > On Sat, 06 May 2017, to...@tuxteam.de wrote: > > Hm. That just means that the file will change after being copied. > > There can also exist busy file *sets*, which need to

Re: Live Fille System Backup

2017-05-06 Thread Henrique de Moraes Holschuh
On Sat, 06 May 2017, to...@tuxteam.de wrote: > Hm. That just means that the file will change after being copied. There can also exist busy file *sets*, which need to be atomically snapshotted (this is, of course, application specific). "busy", as far as backups go, means: 1. a file changes [eith

Re: Live Fille System Backup

2017-05-06 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, May 06, 2017 at 06:42:46AM -0400, Dan Ritter wrote: > On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: > > On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > > > it would be naive at best to think that busy files [...]

Re: Live Fille System Backup

2017-05-06 Thread Dan Ritter
On Sat, May 06, 2017 at 07:52:25AM +0200, to...@tuxteam.de wrote: > On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > > it would be naive at best to think that busy files [...] > > What is a "busy" file? Open for writing. -dsr-

Re: Live Fille System Backup

2017-05-05 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Fri, May 05, 2017 at 01:35:35PM -0700, Sergei G wrote: > it would be naive at best to think that busy files [...] What is a "busy" file? regards - -- t -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlkNZJkACgkQBcgs9

Re: Live Fille System Backup

2017-05-05 Thread Henrique de Moraes Holschuh
On Fri, 05 May 2017, John Conover wrote: > Also be advised if "dest/" is a FAT formatted SD card, there is a file > size limit of 4GB, (and the -aAXv argument to rsync(1) may not be > viable to restore the permission/ACL of files/directories.) Using rsync on non-POSIX filesystems is a Bad Idea, it

Re: Live Fille System Backup

2017-05-05 Thread John Conover
Henrique de Moraes Holschuh writes: > On Fri, 05 May 2017, Sergei G wrote: > > I am not convinced that rsync can handle all device type files, all types > > of links and other attributes I don't even know about. I am not convinced > > that anyone actually restored file system using rsync. Does it

Re: Live Fille System Backup

2017-05-05 Thread Bob Weber
On 5/4/17 7:17 PM, Sergei G wrote: > I am running Raspberry PI and I would like to dump full file system without > shutting down the system. One machine runs nginx and another runs > PostgreSQL. I have had a good success with FreeBSD and dump software, because > it is part of the OS and core team

Re: Live Fille System Backup

2017-05-05 Thread Henrique de Moraes Holschuh
On Fri, 05 May 2017, Sergei G wrote: > I am not convinced that rsync can handle all device type files, all types > of links and other attributes I don't even know about. I am not convinced > that anyone actually restored file system using rsync. Does it really work? Yes, it does. Been there, do

Re: Live Fille System Backup

2017-05-05 Thread Henrique de Moraes Holschuh
On Fri, 05 May 2017, Sergei G wrote: > it would be naive at best to think that busy files can be handled at > application level. No, rsync cannot handle the problem of entire file > system dump. Neither it is safe to handle all special file cases. I am > thinking about sparse files (if Linux ha

Re: Live Fille System Backup

2017-05-05 Thread Sergei G
I just realized that busy files are pretty much impossible to deal with no matter what. If DB is in the middle of the transaction, then a "snapshot" based backup will work (we simply loose that transaction data as it is equivalent to crash). Other busy files might be corrupted if those files don'

Re: Live Fille System Backup

2017-05-05 Thread Sergei G
it would be naive at best to think that busy files can be handled at application level. No, rsync cannot handle the problem of entire file system dump. Neither it is safe to handle all special file cases. I am thinking about sparse files (if Linux has those). On Thu, May 4, 2017 at 11:52 PM, D

Re: Live Fille System Backup

2017-05-05 Thread tomas
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, May 04, 2017 at 04:17:46PM -0700, Sergei G wrote: > I am running Raspberry PI and I would like to dump full file system without > shutting down the system. One machine runs nginx and another runs > PostgreSQL. I have had a good success with F

Re: Live Fille System Backup

2017-05-04 Thread Dejan Jocic
On 04-05-17, Sergei G wrote: > That's good to know. > > > Thank you > > > On 5/4/17 6:46 PM, Anders Andersson wrote: > > On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: > > > I would like a backup tool that does not bring a million dependencies with > > > MBs of files. Something that works on

Re: Live Fille System Backup

2017-05-04 Thread Sergei G
That's good to know. Thank you On 5/4/17 6:46 PM, Anders Andersson wrote: On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: I would like a backup tool that does not bring a million dependencies with MBs of files. Something that works on server without X Windows and can send backup to an exte

Re: Live Fille System Backup

2017-05-04 Thread Anders Andersson
On Fri, May 5, 2017 at 1:17 AM, Sergei G wrote: > I would like a backup tool that does not bring a million dependencies with > MBs of files. Something that works on server without X Windows and can send > backup to an externally attached USB drive. Nothing fancy. No network > infrastructure.

Live Fille System Backup

2017-05-04 Thread Sergei G
I am running Raspberry PI and I would like to dump full file system without shutting down the system. One machine runs nginx and another runs PostgreSQL. I have had a good success with FreeBSD and dump software, because it is part of the OS and core team maintains it. However, dump utility is no