[Discuss] Cloud-backup solutions for Linux?

2015-09-23 Thread Rich Braun
What do you use for offsite backup? Here's why I ask: For a few years I've been using CrashPlan as my primary backup, and rsnapshot as a secondary. About once a year, it seems, CrashPlan does something troubling and it's always felt like Linux takes a back-seat to their Windows and Mac platform.

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-23 Thread Bill Cattey
I have finally fully implemented my offsite backup solution. I have two (physically) small removable drives. I use SuperDuper to create a duplicate of my active drive onto a removable drive. I carry it over to a local bank and swap it with the drive currently in the box. I repeat this process e

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-23 Thread Jack Coats
On occasion, I have had to delete a crashplan client, and on windows machines reboot, before installing it again, and re-claiming the same crashplan machine name. Before crashplan, I used a script with tar to backup to Amazon using 'normal' backup, full, followed by various versions of reduced ver

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-23 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Rich Braun > > Here's why I ask: For a few years I've been using CrashPlan as my primary > backup, and rsnapshot as a secondary. What's wrong with rsnapshot? ___ Discuss m

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matt Shields
Check out ownCloud. It let's you run your own cloud based backup service. Matt On Wed, Sep 23, 2015 at 5:21 PM, Rich Braun wrote: > What do you use for offsite backup? > > Here's why I ask: For a few years I've been using CrashPlan as my primary > backup, and rsnapshot as a secondary. > > Abou

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Matt Shields > > Check out ownCloud. It let's you run your own cloud based backup service. Oh god, no. If you're thinking about ownCloud, try Synctuary instead. I probably can't make a statement about ownCloud wit

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matt Shields
So far have not had a single issue. I have a private cloud in AWS that myself and my family sync to using multiple platforms (Mac, Win & Linux). That instance is backed up to S3. Performance is great, computers never have an issue, performance is great. And ownCloud offers a version that's 100% f

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Matt Shields [mailto:m...@mattshields.org] > > So far have not had a single issue.  I repeat the question: What happens if you interrupt the client or network in the middle of a file transfer? What happens if you create a file with a disallowed character in its name? Be sure to md5sum

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
Also, Synctuary and ownCloud are more for sync/sharing/replication. Not really a backup product. To the OP, I would suggest rsnapshot or rsync for his purposes, not so much Synctuary or ownCloud. ___ Discuss mailing list Discuss@blu.org http://lists.blu

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matt Shields
Depending what the person's use case is, sometimes "good enough" is "good enough". I can deal with my wife or one of my kids mistakenly naming something with a bad character, because I only care that they can re-open it on their computer, not on mine. If they can save the file on a mac, and re-sy

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Rich Pieri
On 9/24/2015 6:36 AM, Matt Shields wrote: Check out ownCloud. It let's you run your own cloud based backup service. ownCloud is sync/sharing, not backup. On 9/24/2015 7:02 AM, Edward Ned Harvey (blu) wrote: > Oh god, no. If you're thinking about ownCloud, try Synctuary instead. So are Synctu

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matt Shields
Who says sync/sharing is not a backup? Is the goal a backup not to have two or more copies of your data in different locations? If the datacenter happens to fail, your other copy would be the local one, correct? Swapping backup drives/tapes isn't without it's own problems. What happens if the ba

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Rich Pieri
On 9/24/2015 10:36 AM, Matt Shields wrote: Who says sync/sharing is not a backup? Is the goal a backup not to have two or more copies of your data in different locations? No, it isn't. The goal to be able to recover from a disaster. Having a replica may be a step toward that goal but it is no

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Jack Coats
Syncing is a form of backup IMHO. But I don't like that it is so easy to access. That makes it to easy for ME to fubar my own data. (Dropbox keeping an old version or so for a while has saved me in the past, but it is secondary to its normal function.) I do use syncing. I use a separate backup,

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matthew Gillen
On 09/24/2015 07:02 AM, Edward Ned Harvey (blu) wrote: > What happens if you create a file with a character in its name, that's not > allowed on some other platform? Somewhat related: you cannot create directories on windows named "aux". Spent way too much time once figuring out why the guy wi

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread David Thompson
"Edward Ned Harvey (blu)" writes: >> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On >> Behalf Of Matt Shields >> >> Check out ownCloud. It let's you run your own cloud based backup service. > > Oh god, no. If you're thinking about ownCloud, try Synctuary instead. The first

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Rich Braun
Most of the comments in response to my original post have addressed only a subset of backup requirements: sync current files to offsite storage. Some of the requirements I have include the following: * As each file is updated, I want a new version saved, and I want history available for each file

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Gordon Marx
On Thu, Sep 24, 2015 at 1:03 PM, David Thompson wrote: > The first distinct advantage I see for ownCloud is that it is free > software, while Synctuary appears to be proprietary SaaS, making its > technical advantages moot for those that want to control their own data > and privacy. > >From a pe

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Matthew Gillen > > Somewhat related: you cannot create directories on windows named "aux". > Spent way too much time once figuring out why the guy with a windows > box couldn't check out code from our repo... Woah

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Jack Coats > > Syncing is a form of backup IMHO. The reason why syncing is not a backup, is because if you delete a file, and the deletion gets replicated, you cannot recover the deleted file. Ability to recover d

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Dan Ritter
On Thu, Sep 24, 2015 at 10:12:06AM -0700, Rich Braun wrote: > > Of the alternatives out there, the one that looks most promising in light of > these requirements is SpiderOak. But no one else here has named that one so I > assume it's an obscure vendor that few of us have heard of. > I've inves

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Dan Ritter > > I've investigated SpiderOak. Add to your requirements: > open-source client Never expose your passwords or encryption keys to anyone. This is the reason I personally don't trust spideroak. I personal

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Rich Braun
Edward Net Harvey questions SpiderOak: > I personally feel it's deceptive marketing > "... When accessing your data via the SpiderOak website or a > mobile device, you must enter your password which will then exist in the > SpiderOak server memory for the duration of your browsing session." I'm ac

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Edward Ned Harvey (blu)
> From: Rich Braun [mailto:ri...@pioneer.ci.net] > > I'm actually OK with that one; the above sentence is taken a bit out of > context in that it's describing a particular feature that few of us would > actually ever use. If you browse to your stuff in the web page, or if you install the mobile

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Matt Shields
ownCloud has version control ( https://doc.owncloud.org/server/8.1/user_manual/files/version_control.html), although you need to keep an eye on your server drive size. It will start to purge older versions if your disks exceed 50%. But by your definition this would still be considered a backup.

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-24 Thread Rich Pieri
On 9/24/2015 3:22 PM, Matt Shields wrote: ownCloud has version control ( Version control is not backup. If the disk dies then the versioned files die with it. Also since I never delete from my S3 bucket where I have a nightly sync from ownCloud to, the risk of losing everything is low. I a

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Matt Shields
By your definition Bill's solution would fail your test of what you need in a backup solution. He makes a backup once every couple months, then runs it offsite. He has a risk of losing up to 2 months worth of data in his scenario. He can't get back changes from Monday or Tuesday. Not saying his

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Matt Shields > > What's > sad about this back and forth is that a few people already made up their > minds to dismiss my solution because it doesn't fit their needs or > definition. Most people care about ownCloud d

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Rich Pieri
On 9/25/2015 8:29 AM, Matt Shields wrote: By your definition Bill's solution would fail your test of what you need in a backup solution. He makes a backup once every couple months, then runs it offsite. You are misrepresenting one aspect of his backup plan, the off-site storage aspect, as the

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Daniel Barrett
I use a hybrid, multi-level process and haven't seen anything similar mentioned, so here goes. First, our home computers use RAID1 mirroring to survive a disk failure, and each is on a UPS. For first-level backups, all computers back up to a RAID5 NAS: the Macs by Time Machine, and the Linux box

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Rich Braun
Daniel Barrett detailed his solution: > I use a hybrid, multi-level process and haven't seen anything > similar mentioned ... (We could improve by using an > rsync+symlinks incremental approach.) I have a couple of suggested improvements, since your solution's quite similar to mine. The rsync/sym

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Bill Cattey
I didn't mention it, because it's not part of my "off site" backup, but YES I am also doing time machine backup. Mind you, my Time Machine drive died and I was really slow to replace it, so I DID have precisely the exposure flagged. But I'm better now. :-) -Bill Rich Pieri wrote: On 9/25/

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Kent Borg
As the cloud gets all cool, remember the virtues of off-line backups. There have probably been a lot of folks who maybe had great backups, what were physically redundant and maintained versions back in time, but were all online and susceptible to malware that wants to encrypt and hold files hos

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-25 Thread Rich Pieri
On 9/25/2015 6:14 PM, Kent Borg wrote: As the cloud gets all cool, remember the virtues of off-line backups. Also remember the Code Spaces breach. Their bullet-proof backup plan turned out not be bullet-proof at all. -- Rich P. ___ Discuss mailing

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-26 Thread Jerry Feldman
Lots of great ideas here. The backup system I used for my company before it got IBM'd was: 1. periodic rsnapshots. As rich mentioned, rsnapshot takes advantage of the --rdest feature of rcync, that created a hard link on duplicate files. 2. Automatically backed up from our NY office nightly. They

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-26 Thread Daniel Barrett
On September 25, 2015, Rich Braun wrote: >I have a couple of suggested improvements, since your solution's >quite similar to mine. [...] Thanks so much for taking the time to suggest improvements. rsnapshot seems like a clear win. I am not familiar with unison but it seems like a good idea for mai

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-26 Thread Bill Bogstad
On Thu, Sep 24, 2015 at 1:55 PM, Edward Ned Harvey (blu) wrote: >> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On >> Behalf Of Jack Coats >> >> Syncing is a form of backup IMHO. > > The reason why syncing is not a backup, is because if you delete a file, and > the deletion ge

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-27 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of Daniel Barrett > > One piece I've never fully worked out is backing up the live VM's > (VMware Workstation) running on my Linux box. For VM's, you only have three choices: 1- Install backup software or something i

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-27 Thread Rich Braun
> Edward Ned Harvey (blu) wrote: > For VM's, you only have three choices: My approach changed after I switched most of my VMs to containers (with LXC). I now make sure all user-data is in a mounted volume shared from the host into the client instance. With containers there's no overhead and yo

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-27 Thread Rich Pieri
On 9/26/2015 11:50 PM, Bill Bogstad wrote: Some people would call that an archival system. While it is true that most backup systems allow you to recover deleted/old version of files, it's not clear that is a required part of a backup system in the strictest sense. Still it comes in handy and

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-27 Thread Bill Bogstad
On Sun, Sep 27, 2015 at 8:16 AM, Edward Ned Harvey (blu) wrote: >> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On >> Behalf Of Daniel Barrett >> >> One piece I've never fully worked out is backing up the live VM's >> (VMware Workstation) running on my Linux box. > > For VM's,

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Edward Ned Harvey (blu)
> From: Bill Bogstad [mailto:bogs...@pobox.com] > > > 2- Use a snapshotting filesystem like btrfs or zfs in the host, so the host > > can > replicate the guest storage to another location seamlessly. > > I don't see how this can work in a way that would be useful. > Filesystem snapshots of your

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Bill Bogstad
On Mon, Sep 28, 2015 at 8:24 AM, Edward Ned Harvey (blu) wrote: >> From: Bill Bogstad [mailto:bogs...@pobox.com] >> >> > 2- Use a snapshotting filesystem like btrfs or zfs in the host, so the >> > host can >> replicate the guest storage to another location seamlessly. >> >> I don't see how this c

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Rich Pieri
On 9/28/2015 8:24 AM, Edward Ned Harvey (blu) wrote: something. The guest storage device is a precise snapshot of what the guest storage would have looked like at the instant that the storage snapshot occurred. No, it isn't. Issuing a flush command on the host does not flush guests' buffers so

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Bill Bogstad
On Mon, Sep 28, 2015 at 12:35 PM, Rich Pieri wrote: > On 9/28/2015 8:24 AM, Edward Ned Harvey (blu) wrote: >> >> something. The guest storage device is a precise snapshot of what the >> guest storage would have looked like at the instant that the storage >> snapshot occurred. > > > No, it isn't. I

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Rich Pieri
On 9/28/2015 1:04 PM, Bill Bogstad wrote: I think you are both right. Ed is right because it is a precise image (snapshot) of an albeit inconsistent filesystem. I get that now. He is correct in that his snapshots are approximately what the guests look like when the power is yanked. Whether o

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread John Abreau
When I backup Postgresql databases, I use pg_dumpall to dump the data into a text file; I don't try to backup the binary database files. I'm not familiar with MongoDB, but I would be surprised if it didn't have a similar option to dump its data to a text file. On Mon, Sep 28, 2015 at 1:44 PM, Ri

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Rich Pieri
On 9/28/2015 2:56 PM, John Abreau wrote: When I backup Postgresql databases, I use pg_dumpall to dump the data into a text file; I don't try to backup the binary database files. Right. You need to either dump the database from within the engine or quiesce the database before making a snapshot

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Bill Bogstad
On Mon, Sep 28, 2015 at 3:37 PM, Rich Pieri wrote: > On 9/28/2015 2:56 PM, John Abreau wrote: >> >> I'm not familiar with MongoDB, but I would be surprised if it didn't have >> a >> similar option to dump its data to a text file. > > > Be surprised. MongoDB lacks the tools to do full text dumps. I

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-28 Thread Rich Pieri
On 9/28/2015 10:49 PM, Bill Bogstad wrote: This web page makes for some "fun" reading on how to back up MongoDB: To borrow a meme, one does not simply back up MongoDB. :) -- Rich P. ___ Discuss mailing list Discuss@blu.org http://lists.blu.org/mailma

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-29 Thread Edward Ned Harvey (blu)
> From: Bill Bogstad [mailto:bogs...@pobox.com] > > While some OSes/filesystems handle power interruption well at this > point, it seems to me that there are lots of apps/servers which do not > and which people still need to use. Particularly in a VM environment > where you might be running lega

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-29 Thread Edward Ned Harvey (blu)
> From: Discuss [mailto:discuss-bounces+blu=nedharvey@blu.org] On > Behalf Of John Abreau > > When I backup Postgresql databases, I use pg_dumpall to dump the data into > a text file; I don't try to backup the binary database files. > > I'm not familiar with MongoDB, but I would be surprised

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-29 Thread Chris Markiewicz
On 09/29/2015 10:28 AM, Edward Ned Harvey (blu) wrote: >> From: Bill Bogstad [mailto:bogs...@pobox.com] >> >> While some OSes/filesystems handle power interruption well at this >> point, it seems to me that there are lots of apps/servers which do not >> and which people still need to use. Particu

Re: [Discuss] Cloud-backup solutions for Linux?

2015-09-29 Thread Rich Pieri
On 9/29/2015 10:28 AM, Edward Ned Harvey (blu) wrote: I haven't seen an OS, Filesystem, or a daemon, it at least 15 years, that couldn't gracefully survive a power interruption. Except ownCloud. ;-) I have. More than once. And had to clean up afterwards. More than once. > has these character