Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

I'm taking about BPC itself. Why? Let's take an example.00:30 My nightly rsync script starts to sync pool to off-site storage. This night, there happens to be a lot of new data an it takes time.01:00 BackupPC_Nightly starts cleaning the pool! Rsync is still running!01:15 My nightly rsync script is done syncingThis night I would end up with a corrupt copy since BPC wrote to the pool while I was copying it. One suggestion to prevent BPC to touch the pool while copying it was to stop the BPC service temporarily. Sounds good, but would that lead to new problems for BPC?If BPC never runs between say 00.00 and 01.30, will this prevent certain tasks in the BPC_Nightly to never run for instance?And what if BPC happens to running a cleaning job when I stop it and start my rsync script? Will I then copy a corrupted pool?Many questions. This approach seems fragile to me. Am I right?/Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: B To: backuppc-users@lists.sourceforge.netCC: On Fri, 11 Aug 2017 00:16:27 +0200Hannes Elvemyr  wrote:> Would it be wise to stop/start BackupPC every night? Can it introduce> any problem? What if my copy process (for instance rsync over Internet> to an off-site storage) takes 1 hour, maybe 2 hours some days, well> that interfere with BackupPC_Nightly?Do you speak about BPC itself or it's machine?If BPC, why, as it almost eat nothing while sleeping,if machine (energy spare, I guess), it isn't recommended as electronics and HDz really hate shocks, either electric(start draw more current as you've got to charge capacitors, thisproduce an overall peak) or thermal, especially HDz.Jean-Yves--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdot___BackupPC-users mailing listBackupPC-users@lists.sourceforge.netList:https://lists.sourceforge.net/lists/listinfo/backuppc-usersWiki:http://backuppc.wiki.sourceforge.netProject: http://backuppc.sourceforge.net/--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

Hi!Thanks for great response!tar + copy will of course take a long time and be inefficient due to the amount of data. What about rsync? Is there any reason why the normal benefits of rsync over tar+copy would not apply for the BackupPC v4 pool?Is there anything, any options to the command, to think about when using rsync? (For instance, can I prevent BackupPC to write to the pool during my rsync job is running?)I will have a look at wireguard. Anyone got a clue of performance compared to openvpn?"I think he meant making a snapshot then convey it to he's 2nd site. " - yes, I did :)/Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: Ray Frush <fr...@rams.colostate.edu>To: "General list for user discussion, questions and support" <backuppc-users@lists.sourceforge.net>CC: Hannes-Option 2, Copying the the pool (tar, then copy) to a remote server appears to be viable for small installations, and we've tested that with success.  But at a certain point, if you have a lot of machines, and your pool gets large, that process becomes un-manageable.  (Our pool is about 3.5TiB for just over 100 hosts backed up).  Just  tar-ing the data store up takes most of the day.So, Option 1 starts to look more appealing.   It may be that your primary instance (local) has a more aggressive retention schedule than the remote copy, or other differences like how frequently backups run.   --Ray FrushOn Wed, Aug 9, 2017 at 2:47 PM, Hannes Elvemyr <hanne...@gmail.com> wrote:
  

  
  

  

Hi!
I'm using BackupPC for all my machines and it's great! I would
  now like to protect my BackupPC pool somehow (if my BackupPC
  server crashes, gets stolen or burns up I don't want to loose the
  data). As I see it, I have at least two options:
1. Run a second instance of BackupPC off-site
This of course creates a new second pool, but that could actually
  be an advantage if one of them somehow gets corrupted.
Pros: Two independent pools.
Cons: Complicated setup. I need a VPN between my network and the
  off-site machine to get this to work, which turned out to be more
  complicated than I first thought (my current router does not
  support static routing, so a new router would be the first step).
  I would also need to sync the BackupPC configuration from my main
  BackupPC instance to the second one (if I change some
  configuration on my main instance, I would also like the second
  instance to get that change).
2. Copying the pool and send it off-site
Pros: No need for a second BackupPC instance. Seems to be the
  easier solution if I can find out how to make a reliable copy of
  the pool.
Cons: How to copy the pool? The version 4 documentation says that “In V4, since
  hardlinks are not used permanently, duplicating a V4 pool is much
  easier, allowing remote copying of the pool.”. Sound great, but
  how do I know that BackupPC is not reading/writing to the pool
  during the copying process (maybe some backup is running or
  BackupPC_Nightly could start doing some cleaning). Copying a large
  pool over a bad Internet connection could take hours…
Any thought on this? How do you get redundancy of your BackupPC
  data?
Thanks!
-- 
/Hannes Elvemyr
  

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
-- Time flies like an arrow, but fruit flies like a banana.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

Would it be wise to stop/start BackupPC every night? Can it introduce any problem? What if my copy process (for instance rsync over Internet to an off-site storage) takes 1 hour, maybe 2 hours some days, well that interfere with BackupPC_Nightly? /Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: Alexander Moisseev via BackupPC-users <backuppc-users@lists.sourceforge.net>To: backuppc-users@lists.sourceforge.netCC: Alexander Moisseev <mois...@mezonplus.ru>On 8/9/2017 11:47 PM, Hannes Elvemyr wrote:> Sound great, but how do I know that BackupPC is not reading/writing to the pool during the copying process (maybe some backup is running or BackupPC_Nightly could start doing some cleaning). Copying a large pool over a bad Internet connection could take hours…> Option 1. Stop BackupPC.Option 2. Make a snapshot of the file system.--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdot___BackupPC-users mailing listBackupPC-users@lists.sourceforge.netList:https://lists.sourceforge.net/lists/listinfo/backuppc-usersWiki:http://backuppc.wiki.sourceforge.netProject: http://backuppc.sourceforge.net/--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

Hi!

I'm using BackupPC for all my machines and it's great! I would now like 
to protect my BackupPC pool somehow (if my BackupPC server crashes, gets 
stolen or burns up I don't want to loose the data). As I see it, I have 
at least two options:


1. Run a second instance of BackupPC off-site

This of course creates a new second pool, but that could actually be an 
advantage if one of them somehow gets corrupted.


Pros: Two independent pools.

Cons: Complicated setup. I need a VPN between my network and the 
off-site machine to get this to work, which turned out to be more 
complicated than I first thought (my current router does not support 
static routing, so a new router would be the first step). I would also 
need to sync the BackupPC configuration from my main BackupPC instance 
to the second one (if I change some configuration on my main instance, I 
would also like the second instance to get that change).


2. Copying the pool and send it off-site

Pros: No need for a second BackupPC instance. Seems to be the easier 
solution if I can find out how to make a reliable copy of the pool.


Cons: How to copy the pool? The version 4 documentation says that “In 
V4, since hardlinks are not used permanently, duplicating a V4 pool is 
much easier, allowing remote copying of the pool.”. Sound great, but how 
do I know that BackupPC is not reading/writing to the pool during the 
copying process (maybe some backup is running or BackupPC_Nightly could 
start doing some cleaning). Copying a large pool over a bad Internet 
connection could take hours…


Any thought on this? How do you get redundancy of your BackupPC data?

Thanks!

--
/Hannes Elvemyr

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/