Re: [BackupPC-users] Backup corrupted, impossible to corect it
My trouble-free BackupPC pool is on an unencrypted LZ4-compressed ZFS pool.
The physical devices are SATA SSDs.
Robert Trevellyan
On Wed, Sep 18, 2024 at 12:23 PM G.W. Haywood
wrote:
> Hi there,
>
> On Wed, 18 Sep 2024, Daniel Berteaud wrote:
>
> > But I also triggered it a few years ago on CentOS 8 (don't have the
> > details as I do not have access to this system anymore)
>
> Was there any correspondence on the mailing list at the time? I see
> nothing from your email address. I wonder if any useful details have
> been recorded somewhere. It would especially be interesting to know
> about the filesystem - and in particular if it was encrypted ZFS which
> so far seems to be a common theme, although obviously not conclusive.
>
> You've seen the problem on two different versions of Perl, so I think
> Perl itself might begin to look a less likely candidate, but in both
> cases your versions of BackupPC, BackupPC::XS and rsync-bpc are very
> similar. Although I'm on ext4, here are the versions I'm using:
>
> Debian 'Bullseye' on arm64.
> Perl 5.32.1 - Debian package
> BackupPC 4.3.1)
> BackupPC::XS 0.59 ) Installed from Github sources, February 2020)
> rsync-bpc 3.1.2.1 )
>
> In your more recent versions there are several potentially interesting
> changes from my older versions.
>
> Maybe others who are using ZFS can let us know their versions/sources?
>
> It might also be useful to get some statistics on the populations of
> the files being backed up.
>
> Have you tried for example setting $Conf{XferLogLevel} to 4 or greater
> to see if there's anything interesting in the logs? I'm sure there
> are other logging-type things which could help narrow this down.
>
> I'd be keen to see the problem reproduced on an unencrypted ZFS. :)
>
> --
>
> 73,
> Ged.
>
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/
>
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Backup corrupted, impossible to corect it
On Mon, Sep 16, 2024 at 12:10 PM G.W. Haywood wrote: > Hi there, > > On Mon, 16 Sep 2024, Daniel Berteaud wrote: > > - Le 21 Ao? 24, ? 2:02, backu...@kosowsky.org a ?crit : > > > >> First of all, the corruption seems almost definitely to be a disk > >> issue and not a backuppc issue. > > > > I'm just configuring a new BackupPC v4.4 (on Debian, using packages > > from the repo), ... > > Does ZFS work on Linux now? That's a genuine question, not some way > of trying to goad ZFS lovers. I ask it because, last time I looked, > there were people here of the opinion that not only did it not work on > Linux but that it probably never would. Admittedly it was a while ago: > ZFS on Linux is quite mature and is included with Ubuntu Server and Desktop releases. It works as intended in my experience. See https://zfsonlinux.org/ > 8<-- > Date: Tue, 2 Jun 2009 18:38:52 -0600 > From: dan > Subject: Re: [BackupPC-users] How to use backuppc with TWO HDD > To: ... > ... > ZFS on linux = bad. ZFS is a solaris thing and will be for some time. > someday *BSD will have stable ZFS but I doubt linux ever will. btrfs will > likely be in wide use which will serve many of the same purposes as ZFS. > ... > 8<-- > > > ... encounter this kind of errors systematically. It's not a > > corruption due to some hardware fault, but clearly a bug in > > BackupPC. > > As I've said before, but I'll repeat, I don't have any BackupPC axe to > grind but Mr. Kosowsky and I both think that what you're seeing looks > like filesystem corruption. You seem to be of the opinion that this > is not the explanation, and you might be right, but to convince me (at > least) unless I've missed something in your posts which makes the case > conclusively and to which you can point, then I think it must be up to > you to produce that evidence. > > > ... This pool is used for other stuff (1 relatively busy mariadb > > server in a VM, one Proxmox Backup Server storage pool), and > > everything is working correctly. Only BackupPC is having issues. > > The "everything is working correctly" part troubles me. I don't see > that you have evidence for that. The way I approach the logic of > troubleshooting, I'd say you haven't had problems with everything else > *yet*. And you might never have trouble with anything else, but that > is not the presence of evidence. It is the absence of evidence. > > > Looks like I'm not the only one affected, see > > https://github.com/backuppc/backuppc/issues/494 > > Hmmm. That's on Linux as well - or more strictly, it was in April. > > > I tried 3 times to wipe and start from empty pool, but this comes > > back everytime. So there's a bug in BackupPC. It might only be > > triggered on ZFS (or maybe it's related to the speed on the > > underlying storage, as this one is really the fastest I ever > > used). But there's something. For the first time in more than 18 > > years of using BackupPC, > > I've been running BackupPC for nearly as long as you have, and I've > never seen such errors. But I run ext4. It would take an awful lot > of persuasion and years of testing for me to switch to anything else. > Last time I tried one of the latest and greatest new filesystems, the > whole thing went belly up in days. I vowed never to do that again. > > AFAICT so far, the best we can say is that the combination of BackuPC > and ZFS on Linux might be problematic. I can't say I'm surprised, but > I can say that we really don't yet know where your problem lies. So > far, I only know of two people running BackupPC on ZFS on Linux. [*] > Both have posted to Github issue 494. Unless there's some lurker here > who's keeping very quiet, in my view as I said above only you and that > other person are in a position in which they will be able to collect > evidence to identify BackupPC as the culprit. Evidence, not anecdote. > I count at least four now :-) My BackupsPC/ZFS/Ubuntu Server combo has been working as expected for almost 6 years. Also I'd say that in a previous post you described a whole lot of what > appeared to me to be completely unnecessary thrashing of storage media > in (something like) the interests of reliability. My feeling at that > time was that you were making a rod for your own back and I still feel > that way. I kept quiet at the time on grounds of tranquility on this > list, but I believe that you have made things much more complex than > necessary and I'd like to suggest a test... > > > I think I cannot trust my backups anymore. ... very worrying. > > There we can agree. My sugested test is that you fire up a system > using ext4, don't do anything fancy with it, run it in parallel with > your ZFS version, and then sit back and see what happens. There may > be alternatives, such as ditching Linux and trying Solaris. I could > only take a back seat if you did somet
Re: [BackupPC-users] PingCmd no longer having effect
Unless I'm missing something, this seems to be important: 2024-04-05 15:28:10 Can't find host myhost.com via NS and netbios Have you tried adding an entry to your hosts file just to get past this and allow further troubleshooting? Robert Trevellyan On Fri, Apr 5, 2024 at 3:45 PM Ian via BackupPC-users < backuppc-users@lists.sourceforge.net> wrote: > On 4/5/24 15:25, to...@tuxteam.de wrote: > > On Fri, Apr 05, 2024 at 02:27:55PM -0400, Ian via BackupPC-users wrote: > >> Hi, > >> > >> I've been using BackupPC for many years now, and have used > /usr/bin/true as > >> the ping command [...] > > Are you sure it's supposed to be /usr/bin/true and not just /bin/true? > > > > Cheers > > Pretty sure, 'which true' returns '/usr/bin/true'. Though, on my system > true also exists at /bin/true, so I tried that as well just now. Same > result. > > I've also tried placing scripts in the PingCmd field and they do not > appear to execute. > > Thanks, > Ian > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] restoring pool
It's the nightly tasks that should be disabled to ensure nothing is purged. Robert Trevellyan On Tue, Jan 30, 2024 at 2:56 PM daggs via BackupPC-users < backuppc-users@lists.sourceforge.net> wrote: > Greetings, > > done all the suggested, now I have access to my backups. > the backup is mounted as rw as it is required when the config is on the > same drive > all the backups are disabled, can I assume they won't be deleted? > > Dagg > > > Sent: Friday, January 26, 2024 at 10:59 PM > > From: "Christian Völker via BackupPC-users" < > backuppc-users@lists.sourceforge.net> > > To: "General list for user discussion, questions and support" < > backuppc-users@lists.sourceforge.net> > > Cc: "Christian Völker" > > Subject: Re: [BackupPC-users] restoring pool > > > > Good point. > > I suggest to mount as Read-only. Should work. > > Grüsse > > Christian Völker > > > > > Am 26.01.2024 um 21:29 schrieb backu...@kosowsky.org: > > > > > > I would be careful as running BackupPC could cause certain old backups > > > to expire that you may not want to expire. > > > Plus, if you don't have all the hosts set properly, it could delete > > > pool files that are needed... > > > > > > I would be VERY careful to: > > > 1. Make sure I have my full config restored -- including desired > > > expiry times > > > 2. That all my desired backup hosts are intact > > > 3. That I know what I am doing before starting the daemon and risking > > > auto-deletion > > > > > > Regardless, definitely make a copy/snapshot before proceeding... so > > > you can undo... > > > > > > Jeff > > > > > > > > > Christian Völker via BackupPC-users wrote at about 10:41:33 +0100 on > Friday, January 26, 2024: > > >> Hi, > > >> > > >> does your HDD contain the full structure of a pool (usually the > > >> structure below /var/lib/BackupPC)? If so I'd recommend to add all the > > >> hosts which are visible as directory in pc/-folder. Stop BackupPC, > mount > > >> your hdd to /var/lib/BackupPC and start BackupPC. > > >> > > >> It should recon the structure and match with the hosts configuration. > > >> > > >> /CV > > >> > > >> > > >>> Am 26.01.24 um 09:35 schrieb daggs via BackupPC-users: > > >>> Greetings. > > >>> > > >>> I have a pool of backups in a external hdd, I've seems to have > misplaced the config for them, is there a way to reconfigure backuppc so > they can be used again? > > >>> > > >>> Thanks, > > >>> > > >>> Dagg > > >>> > > >>> > > >>> ___ > > >>> BackupPC-users mailing list > > >>> BackupPC-users@lists.sourceforge.net > > >>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > >>> Wiki:https://github.com/backuppc/backuppc/wiki > > >>> Project: https://backuppc.github.io/backuppc/ > > >> > > >> > > >> > > >> ___ > > >> BackupPC-users mailing list > > >> BackupPC-users@lists.sourceforge.net > > >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > >> Wiki:https://github.com/backuppc/backuppc/wiki > > >> Project: https://backuppc.github.io/backuppc/ > > > > > > > > > ___ > > > BackupPC-users mailing list > > > BackupPC-users@lists.sourceforge.net > > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > > Wiki:https://github.com/backuppc/backuppc/wiki > > > Project: https://backuppc.github.io/backuppc/ > > > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:https://github.com/backuppc/backuppc/wiki > > Project: https://backuppc.github.io/backuppc/ > > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Best Practice to Speedup RSYNC
"The storage is connected via NFS." ^^^ I suspect this is the main bottleneck. Robert Trevellyan On Thu, Sep 28, 2023 at 5:11 PM wrote: > Hi, > > i use BackupPC on Linux with rsync since a while. One Task as a full > backup with 1146496 files and 191547.2MiB of size. This takes 781.5 mins > of time. The CompressLevel is setup to 9. The storage is connected via > NFS. I use latest BackupPC Version on Debian x64. > > I asked myself, if there is some potential to boost it up. Its also > seems that rsync uses single cpu instead of multi-cpu. May its an option > to split the job in smaller jobs so more rsync processes can pallel. > Perhaps i should lower the compression level. But i got no idea if that > makes a difference. > > What are youre experiences and tipps to maby boost rsyn backups? > > Thanks all > > Taste > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Backup Share Content
Mine (4.4.0) works exactly as expected, i.e. each backup is a moment-in-time snapshot. There's clearly something amiss with jbk's setup. Robert Trevellyan On Sat, Jul 29, 2023 at 11:52 AM Paul Fox wrote: > jbk wrote: > >BackupPC-4.4.0-9.el9.x86_64 > > > >I see now this is a bigger problem than I thought initially. I > thought > >that each backup # was a restore point to the state of the share at > the > >time of that backup but instead it is an accumulation of all backups > w/o > >reflecting any of the deletions of files now non existent on the > source > > Surely that's not correct. If I do a restore of a specific backup, > I'd expect to get exactly the files that were present when that backup > was run. If I wanted files that had been deleted before that backup > was run, I would understand that I'd need to restore them from a > previous backup. > > If this isn't the case, then I'm... shocked, I guess. > > paul > =-- > paul fox, p...@foxharp.boston.ma.us (arlington, ma, where it's 85.1 > degrees) > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Backup strategy for rarely-connected clients
Could it be someone's responsibility to fire the machine up at a specific time on a specific day each week, for at least a couple of hours? The thing is, a rarely-on machine presents other challenges, not least of which is the difficulty of keeping the OS and apps updated. Making sure those things are done would also provide an opportunity for backuppc to do its work. Robert Trevellyan On Tue, Jan 31, 2023 at 3:58 PM Kenneth Porter wrote: > My company has a laptop that spends most of its time on a shelf, not in > use. So I'm constantly seeing a daily BackupPC alert in my email that > the machine hasn't been backed up recently. > > How do others deal with this situation, with machines that are rarely > connected to the LAN for backup? > > An additional problem is that the machine might only be connected for a > short period, so the server might miss that window and fail to back it > in the short period it's connected, as it's busy with other backups. > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] need help with client config for mac ventura backuppc
My guess is it's the out-of-date rsync that ships with macOS, and I think
there may be other oddities with the macOS version. The version I built
from source is 3.1.3 and it seems to work fine. I have it in /usr/local/bin
which is earlier in $PATH than /usr/bin.
Robert Trevellyan
On Mon, Jan 23, 2023 at 10:31 AM Samual Flossie wrote:
> Robert - I couldn't get rsync to work. If you see something in the error
> messages in my initial note or steps to diagnose the issue, I will try to
> fix.
> I have rsync with everything else and like to use it... but in this case,
> tar worked and rsync isn't.
>
> On Mon, Jan 23, 2023 at 9:45 AM Robert Trevellyan <
> robert.trevell...@gmail.com> wrote:
>
>> Glad to know you got it working. Any reason you don't want to use rsync?
>> IIRC I built rsync from source because of the issues with the version that
>> ships with macOS.
>>
>> Robert Trevellyan
>>
>>
>> On Sun, Jan 22, 2023 at 7:17 PM Samual Flossie
>> wrote:
>>
>>> Robert - thanks for the suggestion. suggestion works great!
>>>
>>> I got backuppc working with the mac's.
>>> XferMethod: tar
>>> TarShareName: /Users/*theuser*
>>> TarClientPath: /usr/bin/tar
>>> TarClientCmd: $sshPath -q -x -n -l *theuser* $host env LC_ALL=C
>>> $tarPath -c -v -f - -C $shareName+ --totals
>>> TarClientRestoreCmd: $sshPath -q -x -l *theuser* $host env LC_ALL=C
>>> $tarPath -x -p --numeric-owner --same-owner -v -f - -C $shareName+
>>>
>>> On Sun, Jan 22, 2023 at 10:06 AM Robert Trevellyan <
>>> robert.trevell...@gmail.com> wrote:
>>>
>>>> For the ping issue you could try something like:
>>>> $Conf{PingPath} = '/usr/bin/nc'
>>>> $Conf{PingCmd} = '$pingPath -w 1 -z $host 22'
>>>>
>>>> Robert Trevellyan
>>>>
>>>>
>>>> On Sun, Jan 22, 2023 at 9:55 AM Samual Flossie
>>>> wrote:
>>>>
>>>>> I have several new mac's to add to the backups and have not been able
>>>>> to get them backing up. Prefer to use rsync.
>>>>> This is existing backuppc 4.4.0 that has been running for a year+ and
>>>>> successfully works with linux and windows boxes.
>>>>>
>>>>> I can ssh successfully from backuppc server. ssh -l username clientip
>>>>> I set RsyncSshArgs to include $sshPath -l username for that clientIP
>>>>> and the scope of the backup is that user's files.
>>>>>
>>>>> XferMethod is rsync.
>>>>> I tried using the existing mac rsync (/usr/bin/rsync) but
>>>>> get errors in the log:
>>>>>
>>>>> bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0,
>>>>> KeepOldAttribFiles = 0
>>>>> 2023-01-22 09:40:17 full backup started for directory /Users/username
>>>>> 2023-01-22 09:40:19 Got fatal error during xfer (rsync error: error in
>>>>> rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
>>>>> 2023-01-22 09:40:24 Backup aborted (rsync error: error in rsync protocol
>>>>> data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
>>>>> 2023-01-22 09:40:24 Removing empty backup #0
>>>>> 2023-01-22 09:40:24 BackupPC_backupDelete: removing #0
>>>>> 2023-01-22 09:40:24 BackupPC_backupDelete: No prior backup for merge
>>>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate: computing totals for host
>>>>> theClientIP
>>>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate: host theClientIP got 0
>>>>> errors (took 0 secs)
>>>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate total errors: 0
>>>>> 2023-01-22 09:40:24 BackupPC_backupDelete: got 0 errors
>>>>> 2023-01-22 09:40:24 Finished BackupPC_backupDelete, status = 0 (running
>>>>> time: 0 sec)
>>>>>
>>>>> Maybe I've got a flag wrong in RsyncSshArgs or RsyncArgs or the other
>>>>> Full and Inc variables. No overrides.
>>>>>
>>>>>
>>>>> Some googling and saw concerns that rsync version may have
>>>>> compatibility issues. Currently installed /usr/bin/rsync version 2.6.9
>>>>> protocol version 29.
>>>>>
>>>>> I installed updated rsync via brew on one the mac and now
>>>>> have /opt/homebrew/Cellar/rsync/3.2.7/bin/rsync. I set RsyncClientPath
>>>>> to /opt/homebrew/Cel
Re: [BackupPC-users] need help with client config for mac ventura backuppc
Glad to know you got it working. Any reason you don't want to use rsync?
IIRC I built rsync from source because of the issues with the version that
ships with macOS.
Robert Trevellyan
On Sun, Jan 22, 2023 at 7:17 PM Samual Flossie wrote:
> Robert - thanks for the suggestion. suggestion works great!
>
> I got backuppc working with the mac's.
> XferMethod: tar
> TarShareName: /Users/*theuser*
> TarClientPath: /usr/bin/tar
> TarClientCmd: $sshPath -q -x -n -l *theuser* $host env LC_ALL=C $tarPath
> -c -v -f - -C $shareName+ --totals
> TarClientRestoreCmd: $sshPath -q -x -l *theuser* $host env LC_ALL=C
> $tarPath -x -p --numeric-owner --same-owner -v -f - -C $shareName+
>
> On Sun, Jan 22, 2023 at 10:06 AM Robert Trevellyan <
> robert.trevell...@gmail.com> wrote:
>
>> For the ping issue you could try something like:
>> $Conf{PingPath} = '/usr/bin/nc'
>> $Conf{PingCmd} = '$pingPath -w 1 -z $host 22'
>>
>> Robert Trevellyan
>>
>>
>> On Sun, Jan 22, 2023 at 9:55 AM Samual Flossie
>> wrote:
>>
>>> I have several new mac's to add to the backups and have not been able to
>>> get them backing up. Prefer to use rsync.
>>> This is existing backuppc 4.4.0 that has been running for a year+ and
>>> successfully works with linux and windows boxes.
>>>
>>> I can ssh successfully from backuppc server. ssh -l username clientip
>>> I set RsyncSshArgs to include $sshPath -l username for that clientIP and
>>> the scope of the backup is that user's files.
>>>
>>> XferMethod is rsync.
>>> I tried using the existing mac rsync (/usr/bin/rsync) but
>>> get errors in the log:
>>>
>>> bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles
>>> = 0
>>> 2023-01-22 09:40:17 full backup started for directory /Users/username
>>> 2023-01-22 09:40:19 Got fatal error during xfer (rsync error: error in
>>> rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
>>> 2023-01-22 09:40:24 Backup aborted (rsync error: error in rsync protocol
>>> data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
>>> 2023-01-22 09:40:24 Removing empty backup #0
>>> 2023-01-22 09:40:24 BackupPC_backupDelete: removing #0
>>> 2023-01-22 09:40:24 BackupPC_backupDelete: No prior backup for merge
>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate: computing totals for host
>>> theClientIP
>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate: host theClientIP got 0 errors
>>> (took 0 secs)
>>> 2023-01-22 09:40:24 BackupPC_refCountUpdate total errors: 0
>>> 2023-01-22 09:40:24 BackupPC_backupDelete: got 0 errors
>>> 2023-01-22 09:40:24 Finished BackupPC_backupDelete, status = 0 (running
>>> time: 0 sec)
>>>
>>> Maybe I've got a flag wrong in RsyncSshArgs or RsyncArgs or the other Full
>>> and Inc variables. No overrides.
>>>
>>>
>>> Some googling and saw concerns that rsync version may have compatibility
>>> issues. Currently installed /usr/bin/rsync version 2.6.9 protocol version
>>> 29.
>>>
>>> I installed updated rsync via brew on one the mac and now
>>> have /opt/homebrew/Cellar/rsync/3.2.7/bin/rsync. I set RsyncClientPath
>>> to /opt/homebrew/Cellar/rsync/3.2.7/bin/rsync and tried again and see same
>>> error.
>>>
>>> 2023-01-22 09:17:45 Got fatal error during xfer (rsync error: error in
>>> rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
>>> 2023-01-22 09:17:50 Backup aborted (rsync error: error in rsync protocol
>>> data stream (code 12) at io.c(226) [Receiver=3.1.3.0]
>>>
>>>
>>> Since ping is disabled by default on ventura, I alleviate that check by
>>> override $PingCmd to $pingPath -c 1 127.0.0.1. If you have suggestion on
>>> better way to verify mac is online in an office environment, please share.
>>>
>>> Sammy
>>>
>>> ___
>>> BackupPC-users mailing list
>>> BackupPC-users@lists.sourceforge.net
>>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>>> Wiki:https://github.com/backuppc/backuppc/wiki
>>> Project: https://backuppc.github.io/backuppc/
>>>
>> ___
>> BackupPC-users mailing list
>> BackupPC-users@lists.sourceforge.net
>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> Wiki:https://github.com/backuppc/backuppc/wiki
>> Project: https://backuppc.github.io/backuppc/
>>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/
>
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] need help with client config for mac ventura backuppc
For the ping issue you could try something like:
$Conf{PingPath} = '/usr/bin/nc'
$Conf{PingCmd} = '$pingPath -w 1 -z $host 22'
Robert Trevellyan
On Sun, Jan 22, 2023 at 9:55 AM Samual Flossie wrote:
> I have several new mac's to add to the backups and have not been able to
> get them backing up. Prefer to use rsync.
> This is existing backuppc 4.4.0 that has been running for a year+ and
> successfully works with linux and windows boxes.
>
> I can ssh successfully from backuppc server. ssh -l username clientip
> I set RsyncSshArgs to include $sshPath -l username for that clientIP and
> the scope of the backup is that user's files.
>
> XferMethod is rsync.
> I tried using the existing mac rsync (/usr/bin/rsync) but
> get errors in the log:
>
> bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, KeepOldAttribFiles = 0
> 2023-01-22 09:40:17 full backup started for directory /Users/username
> 2023-01-22 09:40:19 Got fatal error during xfer (rsync error: error in rsync
> protocol data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
> 2023-01-22 09:40:24 Backup aborted (rsync error: error in rsync protocol data
> stream (code 12) at io.c(226) [Receiver=3.1.3.0])
> 2023-01-22 09:40:24 Removing empty backup #0
> 2023-01-22 09:40:24 BackupPC_backupDelete: removing #0
> 2023-01-22 09:40:24 BackupPC_backupDelete: No prior backup for merge
> 2023-01-22 09:40:24 BackupPC_refCountUpdate: computing totals for host
> theClientIP
> 2023-01-22 09:40:24 BackupPC_refCountUpdate: host theClientIP got 0 errors
> (took 0 secs)
> 2023-01-22 09:40:24 BackupPC_refCountUpdate total errors: 0
> 2023-01-22 09:40:24 BackupPC_backupDelete: got 0 errors
> 2023-01-22 09:40:24 Finished BackupPC_backupDelete, status = 0 (running time:
> 0 sec)
>
> Maybe I've got a flag wrong in RsyncSshArgs or RsyncArgs or the other Full
> and Inc variables. No overrides.
>
>
> Some googling and saw concerns that rsync version may have compatibility
> issues. Currently installed /usr/bin/rsync version 2.6.9 protocol version
> 29.
>
> I installed updated rsync via brew on one the mac and now
> have /opt/homebrew/Cellar/rsync/3.2.7/bin/rsync. I set RsyncClientPath
> to /opt/homebrew/Cellar/rsync/3.2.7/bin/rsync and tried again and see same
> error.
>
> 2023-01-22 09:17:45 Got fatal error during xfer (rsync error: error in rsync
> protocol data stream (code 12) at io.c(226) [Receiver=3.1.3.0])
> 2023-01-22 09:17:50 Backup aborted (rsync error: error in rsync protocol data
> stream (code 12) at io.c(226) [Receiver=3.1.3.0]
>
>
> Since ping is disabled by default on ventura, I alleviate that check by
> override $PingCmd to $pingPath -c 1 127.0.0.1. If you have suggestion on
> better way to verify mac is online in an office environment, please share.
>
> Sammy
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/
>
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Rclone dropbox mount
Notice the --one-file-system parameter in the RsyncArgs section of the Xfer Settings page. Think about whether you want to delete it, or whether you want to set up more than one RsyncShareName for the host. Robert Trevellyan On Tue, Dec 6, 2022 at 3:32 PM Admin wrote: > thanks for your answer. > > How can i do? > Il 06/12/2022 15:46, Robert Trevellyan ha scritto: > > By default, with version 4.x, the rsync transfer method does not span > filesystems. > > Robert Trevellyan > > > On Tue, Dec 6, 2022 at 6:31 AM admin admin wrote: > >> Good morning >> I have a problem with a folder where I have mount a dropbox account. >> I use rsync to backup the mount point on internal disk. I use debian 11 >> server >> When I browse the backup, the folder is empty. >> Can someone help me? >> >> ___ >> BackupPC-users mailing list >> BackupPC-users@lists.sourceforge.net >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users >> Wiki:https://github.com/backuppc/backuppc/wiki >> Project: https://backuppc.github.io/backuppc/ >> > > > ___ > BackupPC-users mailing listbackuppc-us...@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Rclone dropbox mount
By default, with version 4.x, the rsync transfer method does not span filesystems. Robert Trevellyan On Tue, Dec 6, 2022 at 6:31 AM admin admin wrote: > Good morning > I have a problem with a folder where I have mount a dropbox account. > I use rsync to backup the mount point on internal disk. I use debian 11 > server > When I browse the backup, the folder is empty. > Can someone help me? > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] When does compression and de-duplication happen?
With the rsync transfer method, checksums are computed on the hosts and duplicate files are not transferred. Not sure about the exact sequence with other transfer methods. Robert Trevellyan On Mon, Sep 19, 2022 at 4:38 AM Kenneth Porter wrote: > When backing up a new system that's similar to an existing system, do I > need enough space on the backup media for the entire new system, or just > what's different? Will the entire client be pulled over and then > de-duped, or does that happen as each file is pulled, comparing it to > what's already in the pool? > > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Run command per file before storing in the pool
I tend to agree. It sounds like an image library archive project, not a backup project. Robert Trevellyan On Thu, Feb 17, 2022 at 10:36 AM Greg Harris wrote: > Yet another sideline sitter here. However, here goes with a very > questionable thought. Maybe BackupPC is the wrong tool for this particular > directory/instance? Perhaps something like Amazon Glacier with Cryptomator > is a wiser choice in this one scenario? > > Thanks, > > Greg Harris > > On Feb 17, 2022, at 10:27 AM, backu...@kosowsky.org wrote: > > G.W. Haywood via BackupPC-users wrote at about 13:24:26 + on Thursday, > February 17, 2022: > > Hi there, > > On Thu, 17 Feb 2022, brogeriofernandes wrote: > > I'm wondering if would be possible to run a command just after > client transfers file data but before it's stored in backuppc > pool. My idea is to do an image compression, like jpeg-xl lossless, > instead of the standard zlib one. > > > Have you considered using a compressing filesystem on the server? > > > I think that is the best idea as: > 1. It is transparent to BackupPC > 2. You benefit from all the optimizations of the underlying file > system > 3. No new coding is needed > 4. No need to create special compression cases for different file > types > 5. Compression is automagically multi-threaded and cached/backgrounded > so as to minimally slow down the program (I never "feel" the > overhead of compression on my btrfs/zsd file system). > 5. It's essentially totally reliable > > It's particularly easy for a filesystem like btrfs... where you can > use 'zstd' which is both fast and good compression. > > I would compare the speed and compression ratio of btrfs with 'zstd' > with the speed and compression ratio of your raw lossless jpg > compression. > > > ... more bandwidth-friendly ... compression before transferring to > server ... > > > The data can be compressed on the client by the transfer tools during > the transfer. This can be purely to reduce network load and it can be > independent of any compression (perhaps by a different method) of the > data when it is stored by the server. The compression algorithms for > transfer and storage can be chosen for different reasons. Of course > if it is required to perform multiple compression and/or decompression > steps for each file, the server will have to handle an increased load. > > This can all be more or less transparent to BackupPC. > > -- > > 73, > Ged. > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] MacOS Backup through rsync
It's not necessary to install rsync from elsewhere. You can install the macOS build tools and then build from source. Robert Trevellyan On Mon, Dec 6, 2021 at 9:48 AM Christian Völker wrote: > Hi Robert, > > thanks for suggestions. I agree this might be the culprit here. And not > really related to BackupPC, sorry. > > > Am 06.12.21 um 15:32 schrieb Robert Trevellyan: > > If you run rsync locally on the Mac, do you get a prompt about needing > 'Full Disk Access' or similar? This is necessary for recent versions of > macOS (since Catalina I believe). > > I allowed rsync and sshd full access to the hard disk through system > settings on my Mac. Still no change, though. > Based on this article <https://developer.apple.com/forums/thread/126497>. > > Are you using the rsync that comes with macOS, or did you build the latest > stable release locally? In the past macOS's rsync has been quite out of > date. > > I will not install a tools from somewhere else, sorry. Especially as it > does not be related to the issue. > > Issue still unresolved even after having granted full disk access to rsync > and sshd. > > Thanks again! > > /KNEBB > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] MacOS Backup through rsync
If you run rsync locally on the Mac, do you get a prompt about needing 'Full Disk Access' or similar? This is necessary for recent versions of macOS (since Catalina I believe). Are you using the rsync that comes with macOS, or did you build the latest stable release locally? In the past macOS's rsync has been quite out of date. Robert Trevellyan On Mon, Dec 6, 2021 at 4:40 AM Christian Völker wrote: > Hi all, > > replying to my own, adding an update: > > When I do a simple "rsync -avz user@macbook:/Users/user/ /tmp/test/ I am > getting a "Operation not permitted (1)" on the folders in question. > > *rsync: opendir "/Users/user/Desktop" failed: Operation not permitted (1)* > *rsync: opendir "/Users/**user/.Trash" failed: Operation not permitted > (1)* > *rsync: opendir "/Users/**user/Documents" failed: Operation not permitted > (1)* > *rsync: opendir "/Users/**user/Downloads" failed: Operation not permitted > (1)* > > Anyone having a clue why? It does not seem to be related to xattr as one > folder has such attributes but the other does not have. > > user@macbook ~ % xattr Desktop > user@macbook ~ % xattr D* > Documents: com.apple.macl > > So the Documents folder has extended attributes, but Desktop doies not > have. Instead, Desktop has a "@" sign in ls -l. What does this mean? > user@macbook ~ % ls -lh| grep D > drwx--+ 11 user staff 352B 6 Dez 08:44 Desktop/ > drwx--@ 95 user staff 3,0K 4 Dez 21:57 Documents/ > drwx--+ 9 user staff 288B 6 Dez 08:41 Downloads/ > > Additionally I am struggling because the "Desktop" has a "+" sign but does > not show xattr > > Thanks for hints! > > /KNEBB > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] ZFS very slow with BackupPC_refCountUpdate
Have you maxed out the RAM? ZFS needs lots of RAM for best performance. Robert Trevellyan On Sun, Apr 25, 2021 at 6:01 AM Ghislain Adnet wrote: > hi :) > >i reply to myself, i think i will abandon zfs completly after trying to > tweak it nothing solve the issue. I have incredible slowdown because of it. > On btrfs host i dont have any issue at all. > >i join a graph of zfs cpu usage (all in iowait) and a btrfs one. If > anyone has any idea it would be great :) > > >the mid day spike is all /BackupPC_refCountUpdate , the night is the > backups > > > -- > cordialement, > Ghislain > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:https://github.com/backuppc/backuppc/wiki > Project: https://backuppc.github.io/backuppc/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Which filesystem for external backup drive?
On Fri, Feb 5, 2021 at 2:41 PM Les Mikesell wrote: > On Fri, Feb 5, 2021 at 1:25 PM Robert Trevellyan > wrote: > > > I'm pretty sure my Ubuntu systems are not using FUSE to access ZFS. If > that means I have to trust the lawyers for Canonical as to the legality of > using ZFS on Ubuntu, so be it. > > https://www.theregister.com/2020/01/13/zfs_linux/ > > Torvalds declared: "Don't use ZFS. It's that simple." > > So, there's always freebsd... I'm not seeking "a Torvalds stamp of approval" on my systems. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Which filesystem for external backup drive?
On Fri, Feb 5, 2021 at 1:34 PM Les Mikesell wrote: > On Fri, Feb 5, 2021 at 12:02 PM Robert Trevellyan > wrote: > > > Problematic in what way? It's been silky smooth for me with Ubuntu. > > Because it is not GPL and the kernel has a restriction against linking > anything that can't be distributed under the GPL. So, you either have > the overhead of FUSE or questionable legality. > I'm pretty sure my Ubuntu systems are not using FUSE to access ZFS. If that means I have to trust the lawyers for Canonical as to the legality of using ZFS on Ubuntu, so be it. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Which filesystem for external backup drive?
On Fri, Feb 5, 2021 at 12:53 PM Les Mikesell wrote: > On Fri, Feb 5, 2021 at 11:45 AM Robert Trevellyan > wrote: > > > > > Is there any reason not to choose ZFS? Even without redundancy, you'll > at least know if it's corrupted. > > ZFS on Linux is always problematic because Linus wants it to be. > Ext4 is the path of least resistance and I've always found its tools > to be good at repairs (had trouble with XFS long ago, back when it's > speed of creating/deleting files made it seem to be worth using) Problematic in what way? It's been silky smooth for me with Ubuntu. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Which filesystem for external backup drive?
On Fri, Feb 5, 2021 at 12:34 PM wrote: > G.W. Haywood via BackupPC-users wrote at about 14:44:57 + on Friday, > February 5, 2021: > > Hi there, > > > > On Fri, 5 Feb 2021 Les Mikesell wrote: > > > > > > On Thu, Feb 4, 2021 at 11:05 PM wrote: > > > > > > > ... Snapshots have also saved me when I have run across the > occasional > > > > backuppc gremlin of disappearing files in that I can find the cpool > > > > file and revert it from past snapshots. > > > > > > Are you sure that the disappearing files aren't a quirk of btrfs in > > > the first place? > > > > > :):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):):) > > > > Seriously, I've never seen a BackupPC file "disappear" - but are we not > > in danger of missing the point that the OP is planning that this drive > > will use a USB interface? > > > > Be *very* careful with any 'modern' filesystem if you use a USB > interface. > > I've seen filesystems trashed by USB communication failures, and the > more > > 'modern' the filesystem, the more irretrievable seems to be the failure. > > > > My advice would be don't do that, unless you're comfortable with trashed > > filesystems and it won't matter to you very much when it happens. > > So what filesystem is safest to run over USB? > Is there any reason not to choose ZFS? Even without redundancy, you'll at least know if it's corrupted. ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:https://github.com/backuppc/backuppc/wiki Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] BackupFilesExclude osx catalina suggestions?
Hi Todd,
I backup Macs with /Users as the RsyncShareName. My excludes are mostly for
transient things like logs and caches:
$Conf{BackupFilesExclude} = {
'/Users' => [
'.DS_Store',
'.Trash',
'.bash*',
'.dropbox*',
'.sync*',
'_sophos',
'Album Artwork',
'Cache*',
'CrashReporter',
'EyeTV Archive',
'Google/Chrome',
'GoogleSoftwareUpdate',
'Logs',
'MobileSync',
'Podcasts',
'PubSub',
'Resilio Sync',
'Saved Application State*',
'Shared/adi',
'SyncServices',
'VirtualBox*'
]
};
Are you trying to backup the entire system, including OS and apps? My goal
is only to backup user data in BackupPC. I let Time Machine take care of
the bare metal backup.
Robert Trevellyan
On Sun, Oct 4, 2020 at 12:25 PM Todd Wade wrote:
>
> I can't find anything on the mailing list so I'm guessing its unlikely but
> -
>
> Any chance someone has assembled a solid BackupFilesExclude for osx
> Catalina?
>
> The link here has no backuppc content any more:
>
> https://github.com/backuppc/backuppc/wiki/How-to-make-Backup-using-Rsync-server-on-Mac-OS
>
>
> Its looking like I backed up about 15gb of /private that I know I don't
> need backed up. Its also looking like I should exclude
> /System/Volumes/Data based on what I'm seeing in the Xfer Error logs.
>
> All in all it looks like I backed up about 25gb that I know I don't need
> backed up, and the disk usage seems to be growing, probably backing up
> stuff I'll never need.
>
> related question:
>
> any advice on how to delete a backup for a host so I can start over? I'd
> like to reclaim this wasted disk space. Theres nothing on the wiki(?)
> and the only things google can find are very old stuff that I'm assuming
> applies to backuppc v3.
>
>
>
> If I have to figure it out myself I'll report back but just hoping
> someone else has been down this path and is willing to share experience.
>
> Love BackupPC what amazing software! I've been using it about 6 months
> and its like a dream. I almost started down the Bacula path and I'm so
> glad I went with BackupPC instead.
>
>
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:https://github.com/backuppc/backuppc/wiki
> Project: https://backuppc.github.io/backuppc/
>
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/
Re: [BackupPC-users] Filesystem Recommendation for 100 TB
I've been using ZFS for storage on Ubuntu Server for several years now. Among other things, the server runs two BackupPC 4 instances in LXC containers. One backs up local machines, the other backs up cloud servers. I haven't run into any serious problems. My use case is on a much smaller scale than yours in terms of storage and number of hosts, but the backups in both BPC instances include large numbers of small files. IMO the fact that ZFS has no limit on inodes is one of the attributes that makes is a good choice, aside from the obvious (reliability and scale). Robert Trevellyan On Tue, Apr 28, 2020 at 1:02 PM Andrew Maksymowsky < andrew.maksymow...@sickkids.ca> wrote: > I have no strong preference for either xfs or zfs (our team is comfortable > with either) was mainly just curious to hear about what folks were using > and if they've run into any major issues or found particular file-system > features they really like when coupled with backuppc. > > Most of the files we'll be backing up are fairly small (under a few mb). > We've got a handful of large databases the we'll also be backing up dumps > of. > > Thanks ! > > Andrew > -- > *From:* Robert Trevellyan > *Sent:* April 28, 2020 12:31 PM > *To:* General list for user discussion, questions and support < > backuppc-users@lists.sourceforge.net> > *Subject:* Re: [BackupPC-users] Filesystem Recommendation for 100 TB > > Any reason not to use ZFS? > > Robert Trevellyan > > > On Tue, Apr 28, 2020 at 11:59 AM Andrew Maksymowsky < > andrew.maksymow...@sickkids.ca> wrote: > > Hello, > > I believe the last time this was asked was a few years ago and I was > wondering if anything has changed. > We’ve been running backuppc for a few years and now have new server for it > with 100 TB of space in hardware raid 6 array. > > We’re wondering what the recommended filesystem for backuppc would be on > an array of this size ? > > Our priority is stability over performance. We’ll be running ubuntu as the > operating system. > > Most of the hosts we’re backing up are fairly small (under 1 TB) linux > servers. Right now we’re backing up about 100 servers for a total of around > 20 TB. (We’re expecting the new server to last a few years). > > Thanks ! > > - Andrew > > > > This e-mail may contain confidential, personal and/or health > information(information which may be subject to legal restrictions on use, > retention and/or disclosure) for the sole use of the intended recipient. > Any review or distribution by anyone other than the person for whom it was > originally intended is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete all copies. > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.sourceforge.net_lists_listinfo_backuppc-2Dusers&d=DwMFaQ&c=Sj806OTFwmuG2UO1EEDr-2uZRzm2EPz39TfVBG2Km-o&r=OaOl9vMMs7To5r1WtpsEjuelxox8OhadYKz_ljkjikk&m=ZZbvpGjyfvgQFNeLlhANgseqK5edmy6zAFemZagmyJI&s=j-fkTu3P2645IiQ51moJNlcudiydzcUbb8fS3xz4dtA&e=> > Wiki:http://backuppc.wiki.sourceforge.net > <https://urldefense.proofpoint.com/v2/url?u=http-3A__backuppc.wiki.sourceforge.net&d=DwMFaQ&c=Sj806OTFwmuG2UO1EEDr-2uZRzm2EPz39TfVBG2Km-o&r=OaOl9vMMs7To5r1WtpsEjuelxox8OhadYKz_ljkjikk&m=ZZbvpGjyfvgQFNeLlhANgseqK5edmy6zAFemZagmyJI&s=vK2U46EwlblSGkiFB50k-PrF0uXeK8xH3Mpi1DLwJAU&e=> > Project: http://backuppc.sourceforge.net/ > <https://urldefense.proofpoint.com/v2/url?u=http-3A__backuppc.sourceforge.net_&d=DwMFaQ&c=Sj806OTFwmuG2UO1EEDr-2uZRzm2EPz39TfVBG2Km-o&r=OaOl9vMMs7To5r1WtpsEjuelxox8OhadYKz_ljkjikk&m=ZZbvpGjyfvgQFNeLlhANgseqK5edmy6zAFemZagmyJI&s=ZCV6mOysmij0l3s3oKWO9takKe0gbiyJGGW1rVcBOfw&e=> > > > -- > > This e-mail may contain confidential, personal and/or health > information(information which may be subject to legal restrictions on use, > retention and/or disclosure) for the sole use of the intended recipient. > Any review or distribution by anyone other than the person for whom it was > originally intended is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete all copies. > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Filesystem Recommendation for 100 TB
Any reason not to use ZFS? Robert Trevellyan On Tue, Apr 28, 2020 at 11:59 AM Andrew Maksymowsky < andrew.maksymow...@sickkids.ca> wrote: > Hello, > > I believe the last time this was asked was a few years ago and I was > wondering if anything has changed. > We’ve been running backuppc for a few years and now have new server for it > with 100 TB of space in hardware raid 6 array. > > We’re wondering what the recommended filesystem for backuppc would be on > an array of this size ? > > Our priority is stability over performance. We’ll be running ubuntu as the > operating system. > > Most of the hosts we’re backing up are fairly small (under 1 TB) linux > servers. Right now we’re backing up about 100 servers for a total of around > 20 TB. (We’re expecting the new server to last a few years). > > Thanks ! > > - Andrew > > > > This e-mail may contain confidential, personal and/or health > information(information which may be subject to legal restrictions on use, > retention and/or disclosure) for the sole use of the intended recipient. > Any review or distribution by anyone other than the person for whom it was > originally intended is strictly prohibited. If you have received this > e-mail in error, please contact the sender and delete all copies. > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC rsync skipping a directory
Glad you got it fixed. Robert Trevellyan On Thu, Apr 16, 2020 at 3:39 PM Robert Mares wrote: > Actually, a co-worker just pointed that out to me, so I removed the > --one-file-system argument and that fixed the problem. I had been banging > my head against the wall on that for a couple of hours and just needed a > fresh set of eyes to see what a dummy I was being. > > Thank you for the response. If he hadn't of pointed it out, this would > have set me in the right direction. > > --Robert > > On 4/16/2020 12:27 PM, Robert Trevellyan wrote: > > Is /home/mailonly on a different filesystem? > > Robert Trevellyan > > > On Thu, Apr 16, 2020 at 3:14 PM Robert Mares > wrote: > >> I have been using BackupPC for a few years now. I am migrating from >> CentOS6 to CentOS8. >> >> I have been using BackupPCv3.1.1 and I am installing BackupPCv4.3.2 >> >> When running with rsync-bpc, it is skipping a directories contents, not >> all directories, just one. >> >> For example, I am attempting to backup /home. It will backup the contents >> of /home/user1, /home/user2, /home/user3, but skip the contents of >> /home/mailonly. When I look at the backups, it says the directory >> /home/mailonly is empty. The old system is working fine and not having any >> issues. I am not using any excludes anywhere. >> >> The command it is running appears to be the following (for obvious >> reasons I have changed the hostname to host-to-backup and >> host-to-backup.fqdn.com): >> >> Running: /usr/bin/rsync_bpc --bpc-top-dir /export/data/BackupPC/ >> --bpc-host-name host-to-backup --bpc-share-name /home --bpc-bkup-num 0 >> --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 >> --bpc-bkup-inode0 4686 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ >> -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args >> --numeric-ids --perms --owner --group -D --times --links --hard-links >> --delete --delete-excluded --one-file-system --partial --log-format=log:\ >> %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 >> host-to-backup.fqdn.com:/home/ / >> >> Any suggestions on what to try would be greatly appreciated. >> >> Thank you in advance. >> >> --Robert >> ___ >> BackupPC-users mailing list >> BackupPC-users@lists.sourceforge.net >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users >> Wiki:http://backuppc.wiki.sourceforge.net >> Project: http://backuppc.sourceforge.net/ >> > > > ___ > BackupPC-users mailing listbackuppc-us...@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] BackupPC rsync skipping a directory
Is /home/mailonly on a different filesystem? Robert Trevellyan On Thu, Apr 16, 2020 at 3:14 PM Robert Mares wrote: > I have been using BackupPC for a few years now. I am migrating from > CentOS6 to CentOS8. > > I have been using BackupPCv3.1.1 and I am installing BackupPCv4.3.2 > > When running with rsync-bpc, it is skipping a directories contents, not > all directories, just one. > > For example, I am attempting to backup /home. It will backup the contents > of /home/user1, /home/user2, /home/user3, but skip the contents of > /home/mailonly. When I look at the backups, it says the directory > /home/mailonly is empty. The old system is working fine and not having any > issues. I am not using any excludes anywhere. > > The command it is running appears to be the following (for obvious reasons > I have changed the hostname to host-to-backup and host-to-backup.fqdn.com > ): > > Running: /usr/bin/rsync_bpc --bpc-top-dir /export/data/BackupPC/ > --bpc-host-name host-to-backup --bpc-share-name /home --bpc-bkup-num 0 > --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 > --bpc-bkup-inode0 4686 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ > -l\ root --rsync-path=/usr/bin/rsync --super --recursive --protect-args > --numeric-ids --perms --owner --group -D --times --links --hard-links > --delete --delete-excluded --one-file-system --partial --log-format=log:\ %o\ > %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum --timeout=72000 > host-to-backup.fqdn.com:/home/ / > > Any suggestions on what to try would be greatly appreciated. > > Thank you in advance. > > --Robert > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] keeping permanent backups
You can change the configuration per host, including how older backups are expired. Will that do what you want? Robert Trevellyan On Mon, Mar 2, 2020 at 5:08 AM Graham Seaman wrote: > I'd like to be able to mark some older full backups as 'frozen', not to > be deleted. Is that possible? I can't find any mention of this in the docs. > > Thanks > > Graham > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Force pool cleanup
Did you stop BackupPC before attempting the command? I believe that's required for most, if not all, command-line operations. BackupPC_nightly will run all the nightly tasks. If your goal is a more aggressive cleanup than usual, you'll need to adjust your nightly cleanup configuration before stopping BackupPC and running the command. Check the usage for BackupPC_nightly to find out if you can tell it to skip updating ref counts. Robert Trevellyan On Thu, Dec 19, 2019 at 10:20 AM Gandalf Corvotempesta < gandalf.corvotempe...@gmail.com> wrote: > Il giorno gio 19 dic 2019 alle ore 16:14 Robert Trevellyan > ha scritto: > > You can run BackupPC_nightly from the command line. > > I'm trying with this: > > su backuppc -c "LC_ALL=C /usr/local/backuppc/bin/BackupPC_nightly -r 0 128" > > but it will exit immediatly: > > BackupPC_nightly lock_off > log BackupPC_nightly skipping BackupPC_refCountUpdate > > I don't want to run refCount, just do pool pruning to free up space > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Force pool cleanup
You can run BackupPC_nightly <http://backuppc:1/filemin/download.cgi?file=BackupPC%5Fnightly&path=%2Fusr%2Flocal%2FBackupPC%2Fbin%2F> from the command line. Robert Trevellyan On Thu, Dec 19, 2019 at 10:09 AM Gandalf Corvotempesta < gandalf.corvotempe...@gmail.com> wrote: > Hi to all. > Any command to run manually to force deletion of "expired" files from > pool to free up disk space? > > I'm running the nightly on a very very low schedule , but right now I > have to run it to clean up as much as possible > > Any hint ? > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Does backuppc support bare metal restore?
BackupPC does not support bare metal restore. If you're replacing a working hard drive, some kind of disk imaging tool is the way to go, e.g. Clonezilla Live or Macrium Reflect. If you're replacing a failed hard drive, this will only work if you already have a recent disk image, which you could then use in combination with an up to date BackupPC backup (restore the disk image, then update the data from BackupPC). Robert Trevellyan On Mon, Nov 25, 2019 at 9:11 AM Marek Grossman wrote: > Hello, > I can't find the clear answer on backuppc WEB site, so I ask you > here. > > Does BackupPC support bare metal restore in nowadays or not (last notice > is from 2007)? > > Is there any other way to restore completely Windows 7/10 installation > with more users after, for example, I change HDD? > > Thanks > -- > Mareg > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] chroot failed
Are you using tar via SMB for macOS hosts? Any reason not to use rsync? My conjecture is that whatever runs on the host to cooperate with BackupPC needs Full Disk Access, but I don't know what that would mean for tar/SMB on macOS. It's not clear to me that anything is launched on the host during such a backup, so maybe it's the SMB client that needs Full Disk Access. Is the chroot error occurring on the BackupPC machine, or on the macOS host? When the real BackupPC experts chime in, they typically ask for logs or log extracts to help with diagnosis, so this might be a good time to post an example showing the failure, perhaps with extra logging . Robert Trevellyan On Wed, Oct 16, 2019 at 10:25 AM David Williams < dwilli...@dtw-consulting.com> wrote: > Robert, > > Thanks for that. What commands would I need to give access to though? I > am connecting via smbclient from my Linux server. Here is the > SmbClientFullCmd: > > $smbClientPath \\$host\$shareName $I_option -U $userName -E -N -d 1 -c > tarmode\ full -Tc$X_option - $fileList > > Regards, > _ > *Dave Williams* > > On Oct 16, 2019, 9:41 AM -0400, Robert Trevellyan < > robert.trevell...@gmail.com>, wrote: > > You probably need to make adjustments in System Preferences -> Security & > Privacy -> Privacy -> Full Disk Access (on the client) to give the relevant > commands the access they need. > > Robert Trevellyan > > > On Tue, Oct 15, 2019 at 8:31 PM David Williams < > dwilli...@dtw-consulting.com> wrote: > >> I am pretty sure that this particular issue started after the upgrade to >> macOS Catalina. >> >> My backups are now failing when it comes to backing up my Documents >> directory with the error message of chroot failed. >> >> Can someone help me troubleshoot and get this working again. I did a >> google search but nothing that seem applicable. >> >> Regards, >> _ >> *Dave Williams* >> >> ___ >> BackupPC-users mailing list >> BackupPC-users@lists.sourceforge.net >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users >> Wiki:http://backuppc.wiki.sourceforge.net >> Project: http://backuppc.sourceforge.net/ >> > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] chroot failed
You probably need to make adjustments in System Preferences -> Security & Privacy -> Privacy -> Full Disk Access (on the client) to give the relevant commands the access they need. Robert Trevellyan On Tue, Oct 15, 2019 at 8:31 PM David Williams wrote: > I am pretty sure that this particular issue started after the upgrade to > macOS Catalina. > > My backups are now failing when it comes to backing up my Documents > directory with the error message of chroot failed. > > Can someone help me troubleshoot and get this working again. I did a > google search but nothing that seem applicable. > > Regards, > _ > *Dave Williams* > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Recommended settings for BackupPC v4 on ZFS
Your TL;DR: for ashift refers to ZFS record size, but ashift determines block size (smallest I/O, with 12 meaning 4096 bytes to match common modern hard drive sector size), not record size. Robert Trevellyan On Thu, Sep 12, 2019 at 8:22 AM Alwyn Kik wrote: > I did some research for this a while ago, and currently use the following > setup: https://gist.github.com/Alveel/c8e80aeef208a7e27c9bd50d0023420c > > Of course this is still far from perfect, and I welcome more ideas or > criticism, but perhaps it can help you out :) > > On 9/11/19 10:03 PM, Carl Soderstrom wrote: > > On 09/11 09:40 , Alexander Moisseev via BackupPC-users wrote: > > On 11.09.2019 18:19, Robert Trevellyan wrote: > > I'm letting ZFS do the compression (using the default of LZ4) with BackupPC > handling deduplication. I think you'll find a reasonable consensus that ZFS > compression is always a win for storage space (it will store un-compressible > data unmodified), whereas ZFS deduplication is best avoided in most cases, > mostly due to its high memory usage. It's possible that BackupPC compression > would be tighter than LZ4, > > Actually, on ZFS you are not limited to LZ4, but in ZFS each file block is > compressed independently, that is why in most cases BackupPC compression is > higher, though it depends on data. > > We moved from 77.96G cpool to pool on compressed filesystem recently. Now it > consumes 81.2G, so there is not much difference. > > # zfs get compression,compressratio,recordsize,referenced zroot/bpc/pool > NAMEPROPERTY VALUE SOURCE > zroot/bpc/pool compressiongzip-3local > zroot/bpc/pool compressratio 3.87x - > zroot/bpc/pool recordsize 128K default > zroot/bpc/pool referenced 81,2G - > > Thanks Alexander, those details are really helpful. > > > -- > Alwyn > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Recommended settings for BackupPC v4 on ZFS
One more thing about ZFS in general - I always set noatime. Robert Trevellyan On Tue, Sep 10, 2019 at 10:50 PM Carl Soderstrom < carl.soderst...@real-time.com> wrote: > Thanks for the advice. Anyone else care to share their experience? > > On 09/10 02:57 , Ray Frush wrote: > > We backup to a ZFS based appliance, and we allow ZFS to do compression > and disable compression in BackupPC. We do not allow ZFS to de-duplicate. > However since you’re looking at doing ZFS on the same box that’s running > BackupPC it probably doesn’t matter which one you have compression turned > on since it’s the same processors., just don’t do both. > > > > > > > > > > > > > On Sep 10, 2019, at 12:26, Carl Soderstrom < > carl.soderst...@real-time.com> wrote: > > > > > > We've been a BackupPC v3 shop since there's been a v3 and we're > looking at > > > building our first v4 BackupPC server. The boss wants to put it on ZFS > and a > > > JBOD controller. > > > > > > I believe that for BackupPC v3 the advice was to turn off ZFS > > > filesystem-level deduplication and compression. > > > > > > Is that still true for BackupPC v4? > > > Are there any other suggestions for filesystem settings for ZFS when > hosting > > > a BackupPC pool? > > > > > > -- > > > Carl Soderstrom > > > Systems Administrator > > > Real-Time Enterprises > > > www.real-time.com > > > > > > > > > ___ > > > BackupPC-users mailing list > > > BackupPC-users@lists.sourceforge.net > > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > > Wiki:http://backuppc.wiki.sourceforge.net > > > Project: http://backuppc.sourceforge.net/ > > > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:http://backuppc.wiki.sourceforge.net > > Project: http://backuppc.sourceforge.net/ > > -- > Carl Soderstrom > Systems Administrator > Real-Time Enterprises > www.real-time.com > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Recommended settings for BackupPC v4 on ZFS
I'm letting ZFS do the compression (using the default of LZ4) with BackupPC handling deduplication. I think you'll find a reasonable consensus that ZFS compression is always a win for storage space (it will store un-compressible data unmodified), whereas ZFS deduplication is best avoided in most cases, mostly due to its high memory usage. It's possible that BackupPC compression would be tighter than LZ4, but I'm not short of storage space for BackupPC so I haven't compared the two. One frequently overlooked benefit of enabling ZFS compression is that, when enabled, regardless of scheme, it leaves all-zero blocks unallocated. Robert Trevellyan On Tue, Sep 10, 2019 at 10:50 PM Carl Soderstrom < carl.soderst...@real-time.com> wrote: > Thanks for the advice. Anyone else care to share their experience? > > On 09/10 02:57 , Ray Frush wrote: > > We backup to a ZFS based appliance, and we allow ZFS to do compression > and disable compression in BackupPC. We do not allow ZFS to de-duplicate. > However since you’re looking at doing ZFS on the same box that’s running > BackupPC it probably doesn’t matter which one you have compression turned > on since it’s the same processors., just don’t do both. > > > > > > > > > > > > > On Sep 10, 2019, at 12:26, Carl Soderstrom < > carl.soderst...@real-time.com> wrote: > > > > > > We've been a BackupPC v3 shop since there's been a v3 and we're > looking at > > > building our first v4 BackupPC server. The boss wants to put it on ZFS > and a > > > JBOD controller. > > > > > > I believe that for BackupPC v3 the advice was to turn off ZFS > > > filesystem-level deduplication and compression. > > > > > > Is that still true for BackupPC v4? > > > Are there any other suggestions for filesystem settings for ZFS when > hosting > > > a BackupPC pool? > > > > > > -- > > > Carl Soderstrom > > > Systems Administrator > > > Real-Time Enterprises > > > www.real-time.com > > > > > > > > > ___ > > > BackupPC-users mailing list > > > BackupPC-users@lists.sourceforge.net > > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > > Wiki:http://backuppc.wiki.sourceforge.net > > > Project: http://backuppc.sourceforge.net/ > > > > > > > > ___ > > BackupPC-users mailing list > > BackupPC-users@lists.sourceforge.net > > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > > Wiki:http://backuppc.wiki.sourceforge.net > > Project: http://backuppc.sourceforge.net/ > > -- > Carl Soderstrom > Systems Administrator > Real-Time Enterprises > www.real-time.com > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] RsyncClientCmd
I think what you're looking for is under Edit Config -> Xfer. Robert Trevellyan On Tue, Apr 30, 2019 at 11:29 AM Stefan Schumacher < stefan.schumac...@net-federation.de> wrote: > Hello, > > in BackupPC 3 i used a modified RsyncClientCmd to nice and ionice rsync > on the backup hosts so that it would stay responsive even when being > backupped and under webserver load at the same time. The variable is > called RsyncClientCmd:$sshPath -q -x -l root $host /usr/bin/nice -n 19 > /usr/bin/ionice -c 3 $rsyncPath $argList+ > > Where should I add these two extra commands in BackupPC4? It seems > RsyncClientCmd does not exist anymore. > > Yours sincerely > Stefan > > > Stefan Schumacher > Systemadministrator > > NetFederation GmbH > Sürther Hauptstraße 180 B - > Fon:+49 (0)2236/3936-701 > > E-Mail: stefan.schumac...@net-federation.de > Internet: http://www.net-federation.de > Besuchen Sie uns doch auch auf facebook, twitter, Google+, flickr, > Slideshare, XING oder unserem Blog. Wir freuen uns! > > > *** > Unser MR Benchmark 2019 ist erschienen! > *** > > Wer hat das beste Media Center und wie sieht die digitale Pressearbeit bei > großen Unternehmen insgesamt aus? > > Die Ergebnisse und das Ranking finden Sie in unserem neuen MR Benchmark > unter: > https://www.mr-benchmark.de/media-relations-benchmark-2019/benchmark > * > > NetFederation GmbH > Geschäftsführung: Christian Berens, Thorsten Greiten > Amtsgericht Köln, HRB Nr. 32660 > > * > > The information in this e-mail is confidential and may be legally > privileged. It is intended solely for the addressee and access to the > e-mail by anyone else is unauthorised. If you are not the intended > recipient, any disclosure, copying, distribution or any action taken or > omitted to be taken in reliance on it, is prohibited and may be unlawful. > If you have received this e-mail in error please forward to: > post...@net-federation.de > > > Die in dieser E-Mail enthaltenen Informationen sind vertraulich und können > von rechtlicher Bedeutung sein. Diese Mail ist ausschließlich für den > Adressaten bestimmt und jeglicher Zugriff durch andere Personen ist nicht > zulässig. Falls Sie nicht der beabsichtigte Empfänger sind, ist jegliche > Veröffentlichung, Vervielfältigung, Verteilung oder sonstige in diesem > Zusammenhang stehende Handlung untersagt und unter Umständen ungesetzlich. > Falls Sie diese E-Mail irrtümlich erhalten haben, leiten Sie sie bitte > weiter an: post...@net-federation.de > > > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Compression
3. Sorry, I think of the machines being backed up as clients, but BackupPC does call them hosts. rsync supports compressed transfers but that's not the scheme used for storage by BackupPC. 4. You may be thinking of the tasks that check for unreferenced files and recalculate the total pool size, which can both be split over multiple nightly runs. Robert Trevellyan On Sat, Jan 12, 2019 at 11:04 AM Jan Stransky wrote: > Hi Robert, > > 1-2) This is what I would expect, I am currious if there is a way to > gradually compress the files; not all at once. > > 3) By the host, I meant host being backed up. And I am sure, it is not > used for the compression, unless compress option of rsync is used. But I > guess, this is uncompressed and compressed again on the server side. The > think is that Pentium N3700 is not very efficient worker, however i7, which > is backed up is different story > > 4) I think there is an option how big percentage of files is being checked > or so; what about this one? > > Cheers, > > Jan > Dne 12.01.2019 v 14:55 Robert Trevellyan napsal(a): > > Hi Jan, > > I think this is correct, but there are other experts who might chime in to > correct me. > > 1. Migration will not result in compression of existing backups. It just > allows V4 to consume the V3 pool. > > 2. After compression is turned on, newly backed up files will be > compressed. Existing backups will remain as they were. > > 3. Host resources are always used for compression, either because BackupPC > is doing the work, or because the filesystem does it natively. My BackupPC > pools are on ZFS with LZ4 compression enabled and BackupPC compression > disabled. > > 4. The closest thing would be using rsync with --ignore-times for full > backups, which isn't quite the same thing, or having the filesystem do it > (e.g. ZFS scrub). > > Robert Trevellyan > > > On Sat, Jan 12, 2019 at 7:51 AM Jan Stransky > wrote: > >> Hi, >> >> I have few questions related to compression >> >> Currently, I have BackupPC 3 installed on Intel NUC with 4 core pentium, >> and since the compression significantly decreased backup speeds, I have >> turned it off. I am about to switch to v4, so it might be worth to >> reconsider, since the increments are not big, big data are already in >> the pool and v4 handles files differently. >> >> 1) If I start the migration with compression on, when would it happen? >> Would whole pool be compressed at once? >> >> 2) If the pool is uncompressed, and I turned it on, when will be >> uncompressed files compressed? On full backup, or never unless ti >> changes on the host? >> >> 3) Is there way to compress the files using the host resources? >> >> 4) Is integrity of the compressed file checked any time? >> >> Best regards, >> >> Jan >> >> >> >> ___ >> BackupPC-users mailing list >> BackupPC-users@lists.sourceforge.net >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users >> Wiki:http://backuppc.wiki.sourceforge.net >> Project: http://backuppc.sourceforge.net/ >> > > > ___ > BackupPC-users mailing listbackuppc-us...@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Compression
Hi Jan, I think this is correct, but there are other experts who might chime in to correct me. 1. Migration will not result in compression of existing backups. It just allows V4 to consume the V3 pool. 2. After compression is turned on, newly backed up files will be compressed. Existing backups will remain as they were. 3. Host resources are always used for compression, either because BackupPC is doing the work, or because the filesystem does it natively. My BackupPC pools are on ZFS with LZ4 compression enabled and BackupPC compression disabled. 4. The closest thing would be using rsync with --ignore-times for full backups, which isn't quite the same thing, or having the filesystem do it (e.g. ZFS scrub). Robert Trevellyan On Sat, Jan 12, 2019 at 7:51 AM Jan Stransky wrote: > Hi, > > I have few questions related to compression > > Currently, I have BackupPC 3 installed on Intel NUC with 4 core pentium, > and since the compression significantly decreased backup speeds, I have > turned it off. I am about to switch to v4, so it might be worth to > reconsider, since the increments are not big, big data are already in > the pool and v4 handles files differently. > > 1) If I start the migration with compression on, when would it happen? > Would whole pool be compressed at once? > > 2) If the pool is uncompressed, and I turned it on, when will be > uncompressed files compressed? On full backup, or never unless ti > changes on the host? > > 3) Is there way to compress the files using the host resources? > > 4) Is integrity of the compressed file checked any time? > > Best regards, > > Jan > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Are Files Re-Copied In Pool On Reinstall of Backup Client Operating System
Which transfer method are you using? With rsync or rsyncd, the data will not be copied again. Robert Trevellyan On Tue, Jun 19, 2018 at 10:46 AM, Akibu Flash wrote: > I am using BackupPC 4 on an ArchLinux distribution to backup a Windows > client. The amount of data on the Windows client being backed up is about > 3.7 terabytes in the BackupPC pool. I would like to reinstall the > operating system on my Windows computer and would like to know the > following: > > 1) Once I reinstall the operating system and direct BackupPC to resume > backing up that Windows client, will BackupPC re-copy all of the data from > the Windows client into the pool? > > 2) On resuming backing up the Windows client, will BackupPC basically > first look at the data to be backed up on the Windows client and then > realize that it already has that data in the pool and therefore will not > re-copy the data but will link to it in the pool. > > The reason I ask re re-copying the data is because the ArchLinux server > only has about 5 terabytes of space available of which 3.7 terabytes is > already filled with backups from the old Windows client and thus if it > re-copies the data from the reinstalled Windows client (even if it later > de-duplicates the data), the ArchLinux server will run out of space. > > Thanks in advance. > > Akibu > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] how can I identify in which backup a certain file is?
Perhaps scouring the XferLOG for lines marked 'new' would help? Robert Trevellyan On Tue, May 15, 2018 at 1:54 AM, Michael Schumacher < michael.schumac...@pamas.de> wrote: > > > RT> Not sure if this is helpful, but if you browse to a specific > RT> folder for a host, you can then click a link to view that folder's > history. > > thanks Robert, I was aware of that feature, but it only allows a per > directory browsing. With a typical Windows directory structure, it is > impossible to identify the one hidden directory that causes the > trouble. > > What I have in mind is either a listing what files are actually new or > some kind of display that would only show that part of the directory > structure that was actually modified in a given backup. > > > best regards > --- > Michael Schumacher > PAMAS Partikelmess- und Analysesysteme GmbH > Dieselstr.10, D-71277 Rutesheim > Tel +49-7152-99630 > Fax +49-7152-996333 > Geschäftsführer: Gerhard Schreck > Handelsregister B Stuttgart HRB 252024 > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] how can I identify in which backup a certain file is?
Not sure if this is helpful, but if you browse to a specific folder for a host, you can then click a link to view that folder's history. Robert Trevellyan On Mon, May 14, 2018 at 2:56 AM, Michael Schumacher < michael.schumac...@pamas.de> wrote: > good morning, > > one of the windows clients that are backed up shows quite large number > and size of new files in every backup. I suspect that some temp files > are included over and over again. > > As far as I can see, there is no way to identify a list of files that > have actually been altered and that are included in a certain > incremental backup. This would be necessary to exclude that temp file > from backup. > > Did I miss something or is that actually not possible. > > best regards > --- > Michael Schumacher > PAMAS Partikelmess- und Analysesysteme GmbH > Dieselstr.10, D-71277 Rutesheim > Tel +49-7152-99630 > Fax +49-7152-996333 > Geschäftsführer: Gerhard Schreck > Handelsregister B Stuttgart HRB 252024 > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] about migrate to a new server
> > On Thu, May 10, 2018 at 2:37 PM, Luigi Augello > wrote: > > > > I cannot preserve the old server I will return it because it was on > leasing. > Can you purchase the hard drive(s) out of the server? -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
That's actually a slightly different error (io.c(1556) instead of io.c(629)). It's possible that browsing the source code of io.c would offer a clue, but it's a pretty generic error, so it's probably time to enable verbose logging on both ends and do some proper debugging. I'd do that by running rsync at the command-line. Robert Trevellyan On Tue, Mar 13, 2018 at 11:13 AM, Marc Gilliatt wrote: > I've set the timeout on my backuppc server to 604800 which is one week. > One of my clients has just completed its first full backup. However, my > other clients are still receiving the same rysnc error as before... > > > 2018-03-13 15:11:29 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(1556) [generator=3.0.9.11]) > 2018-03-13 15:11:34 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(1556) [generator=3.0.9.11]) > > > -- > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:44:54 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > I'd say its worth a try. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 10:30 AM, Marc Gilliatt > wrote: > > So I should set my ConnectTimeout on my backuppc server then? > -- > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:25:30 > > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > According to the man pages, the ConnectTimeout and ServerAlive settings > only apply to ssh_config, i.e. the BPC end, whereas the other settings only > apply to sshd_config, i.e. the host you are trying to backup. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 10:12 AM, Marc Gilliatt > wrote: > > Thank you for the quick response. > > > Do I have to increase the SSH timeout on my backuppc server or my client > or both? > > > I've checked the SSH timeout on my backuppc server, its commented out, > should I uncomment it and set it? > > > > -- > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:00:17 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Maybe the client has its own firewall? > > The BPC timeout setting is ClientTimeout on the Backup Settings page. For > SSH you would have to delve into /etc/ssh_config and /etc/sshd_config. > Look for ConnectionTimeout, ClientAliveInterval, ClientAliveCountMax and > ServerAliveInterval. More details via man ssh_config and man sshd_config. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 9:51 AM, Marc Gilliatt > wrote: > > Our servers our on our internal LAN, so I don't think it's our firewall. I > used BackupPC v3 in the past, and I upgraded to BackupPC v4. I had issues > with the upgrade and I had to completely build my backuppc server again. > However, I had no issue with BackupPC v3, I never received that rsync error > before until now. When I go into each of my client's page in the GUI, it > says its been successfully pinged. > > > I'm not sure on how to find out about the timeout and setting it greater, > or the SSH connection time sorry? > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 21:38:11 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Perhaps the client's firewall is blocking pings. > > And what do you have set for timeout - is it 600 seconds? Or perhaps the > SSH connection is timing out. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 12:46 PM, Marc Gilliatt > wrote: > > I believe I solved my issue of the hostname, I had to use the IP address > for some reason, I cant use the FQDN. > > > Also, when I try to run a backup it fails after about 10 minutes, I'm > receiving the following message... > > > 2018-03-12 16:39:08 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.11]) > 2018-03-12 16:39:14 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(629) [generator=3.0.9.11]) > > I'm not sure on what that error means? > > -- > *From:* Stefan Peter > *Sent:* 12 March 2018 15:54:53 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: [Backu
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
I'd say its worth a try. Robert Trevellyan On Tue, Mar 13, 2018 at 10:30 AM, Marc Gilliatt wrote: > So I should set my ConnectTimeout on my backuppc server then? > -- > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:25:30 > > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > According to the man pages, the ConnectTimeout and ServerAlive settings > only apply to ssh_config, i.e. the BPC end, whereas the other settings only > apply to sshd_config, i.e. the host you are trying to backup. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 10:12 AM, Marc Gilliatt > wrote: > > Thank you for the quick response. > > > Do I have to increase the SSH timeout on my backuppc server or my client > or both? > > > I've checked the SSH timeout on my backuppc server, its commented out, > should I uncomment it and set it? > > > > -- > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:00:17 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Maybe the client has its own firewall? > > The BPC timeout setting is ClientTimeout on the Backup Settings page. For > SSH you would have to delve into /etc/ssh_config and /etc/sshd_config. > Look for ConnectionTimeout, ClientAliveInterval, ClientAliveCountMax and > ServerAliveInterval. More details via man ssh_config and man sshd_config. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 9:51 AM, Marc Gilliatt > wrote: > > Our servers our on our internal LAN, so I don't think it's our firewall. I > used BackupPC v3 in the past, and I upgraded to BackupPC v4. I had issues > with the upgrade and I had to completely build my backuppc server again. > However, I had no issue with BackupPC v3, I never received that rsync error > before until now. When I go into each of my client's page in the GUI, it > says its been successfully pinged. > > > I'm not sure on how to find out about the timeout and setting it greater, > or the SSH connection time sorry? > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 21:38:11 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Perhaps the client's firewall is blocking pings. > > And what do you have set for timeout - is it 600 seconds? Or perhaps the > SSH connection is timing out. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 12:46 PM, Marc Gilliatt > wrote: > > I believe I solved my issue of the hostname, I had to use the IP address > for some reason, I cant use the FQDN. > > > Also, when I try to run a backup it fails after about 10 minutes, I'm > receiving the following message... > > > 2018-03-12 16:39:08 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.11]) > 2018-03-12 16:39:14 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(629) [generator=3.0.9.11]) > > I'm not sure on what that error means? > > -- > *From:* Stefan Peter > *Sent:* 12 March 2018 15:54:53 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > On 12.03.2018 15:09, Marc Gilliatt wrote: > > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > > > > I copied and pasted the RSA key from my backuppc server to my host I > > would like backing up. I ran the following commands, > > > > > > On my backuppc server: > > > > /cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy > > / > > > > > > On my host server: > > > > ///cat id_rsa.pub_copy >> /root/.ssh/authorized_keys/ > > > > / > > / > > I don't see any copy command from the backuppc server to the client here? > > Why dont you use > > ssh-copy-id -i /var/lib/backuppc/.ssh/id_rsa.pub root@backuppc-client > > (execute this as root user on the backuppc server!) > > > With kind regards > > Stefan Peter > > > -- > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing in e-mail
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
According to the man pages, the ConnectTimeout and ServerAlive settings only apply to ssh_config, i.e. the BPC end, whereas the other settings only apply to sshd_config, i.e. the host you are trying to backup. Robert Trevellyan On Tue, Mar 13, 2018 at 10:12 AM, Marc Gilliatt wrote: > Thank you for the quick response. > > > Do I have to increase the SSH timeout on my backuppc server or my client > or both? > > > I've checked the SSH timeout on my backuppc server, its commented out, > should I uncomment it and set it? > > > > ------ > *From:* Robert Trevellyan > *Sent:* 13 March 2018 14:00:17 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Maybe the client has its own firewall? > > The BPC timeout setting is ClientTimeout on the Backup Settings page. For > SSH you would have to delve into /etc/ssh_config and /etc/sshd_config. > Look for ConnectionTimeout, ClientAliveInterval, ClientAliveCountMax and > ServerAliveInterval. More details via man ssh_config and man sshd_config. > > Robert Trevellyan > > On Tue, Mar 13, 2018 at 9:51 AM, Marc Gilliatt > wrote: > > Our servers our on our internal LAN, so I don't think it's our firewall. I > used BackupPC v3 in the past, and I upgraded to BackupPC v4. I had issues > with the upgrade and I had to completely build my backuppc server again. > However, I had no issue with BackupPC v3, I never received that rsync error > before until now. When I go into each of my client's page in the GUI, it > says its been successfully pinged. > > > I'm not sure on how to find out about the timeout and setting it greater, > or the SSH connection time sorry? > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 21:38:11 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Perhaps the client's firewall is blocking pings. > > And what do you have set for timeout - is it 600 seconds? Or perhaps the > SSH connection is timing out. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 12:46 PM, Marc Gilliatt > wrote: > > I believe I solved my issue of the hostname, I had to use the IP address > for some reason, I cant use the FQDN. > > > Also, when I try to run a backup it fails after about 10 minutes, I'm > receiving the following message... > > > 2018-03-12 16:39:08 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.11]) > 2018-03-12 16:39:14 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(629) [generator=3.0.9.11]) > > I'm not sure on what that error means? > > -- > *From:* Stefan Peter > *Sent:* 12 March 2018 15:54:53 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > On 12.03.2018 15:09, Marc Gilliatt wrote: > > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > > > > I copied and pasted the RSA key from my backuppc server to my host I > > would like backing up. I ran the following commands, > > > > > > On my backuppc server: > > > > /cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy > > / > > > > > > On my host server: > > > > ///cat id_rsa.pub_copy >> /root/.ssh/authorized_keys/ > > > > / > > / > > I don't see any copy command from the backuppc server to the client here? > > Why dont you use > > ssh-copy-id -i /var/lib/backuppc/.ssh/id_rsa.pub root@backuppc-client > > (execute this as root user on the backuppc server!) > > > With kind regards > > Stefan Peter > > > -- > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing in e-mail? > (See https://en.wikipedia.org/wiki/Posting_style for details) > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net >
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
Maybe the client has its own firewall? The BPC timeout setting is ClientTimeout on the Backup Settings page. For SSH you would have to delve into /etc/ssh_config and /etc/sshd_config. Look for ConnectionTimeout, ClientAliveInterval, ClientAliveCountMax and ServerAliveInterval. More details via man ssh_config and man sshd_config. Robert Trevellyan On Tue, Mar 13, 2018 at 9:51 AM, Marc Gilliatt wrote: > Our servers our on our internal LAN, so I don't think it's our firewall. I > used BackupPC v3 in the past, and I upgraded to BackupPC v4. I had issues > with the upgrade and I had to completely build my backuppc server again. > However, I had no issue with BackupPC v3, I never received that rsync error > before until now. When I go into each of my client's page in the GUI, it > says its been successfully pinged. > > > I'm not sure on how to find out about the timeout and setting it greater, > or the SSH connection time sorry? > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 21:38:11 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Perhaps the client's firewall is blocking pings. > > And what do you have set for timeout - is it 600 seconds? Or perhaps the > SSH connection is timing out. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 12:46 PM, Marc Gilliatt > wrote: > > I believe I solved my issue of the hostname, I had to use the IP address > for some reason, I cant use the FQDN. > > > Also, when I try to run a backup it fails after about 10 minutes, I'm > receiving the following message... > > > 2018-03-12 16:39:08 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.11]) > 2018-03-12 16:39:14 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(629) [generator=3.0.9.11]) > > I'm not sure on what that error means? > > -- > *From:* Stefan Peter > *Sent:* 12 March 2018 15:54:53 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > On 12.03.2018 15:09, Marc Gilliatt wrote: > > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > > > > I copied and pasted the RSA key from my backuppc server to my host I > > would like backing up. I ran the following commands, > > > > > > On my backuppc server: > > > > /cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy > > / > > > > > > On my host server: > > > > ///cat id_rsa.pub_copy >> /root/.ssh/authorized_keys/ > > > > / > > / > > I don't see any copy command from the backuppc server to the client here? > > Why dont you use > > ssh-copy-id -i /var/lib/backuppc/.ssh/id_rsa.pub root@backuppc-client > > (execute this as root user on the backuppc server!) > > > With kind regards > > Stefan Peter > > > -- > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing in e-mail? > (See https://en.wikipedia.org/wiki/Posting_style for details) > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
Perhaps the client's firewall is blocking pings. And what do you have set for timeout - is it 600 seconds? Or perhaps the SSH connection is timing out. Robert Trevellyan On Mon, Mar 12, 2018 at 12:46 PM, Marc Gilliatt wrote: > I believe I solved my issue of the hostname, I had to use the IP address > for some reason, I cant use the FQDN. > > > Also, when I try to run a backup it fails after about 10 minutes, I'm > receiving the following message... > > > 2018-03-12 16:39:08 Got fatal error during xfer (rsync error: error in rsync > protocol data stream (code 12) at io.c(629) [generator=3.0.9.11]) > 2018-03-12 16:39:14 Backup aborted (rsync error: error in rsync protocol data > stream (code 12) at io.c(629) [generator=3.0.9.11]) > > I'm not sure on what that error means? > > -- > *From:* Stefan Peter > *Sent:* 12 March 2018 15:54:53 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > On 12.03.2018 15:09, Marc Gilliatt wrote: > > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > > > > I copied and pasted the RSA key from my backuppc server to my host I > > would like backing up. I ran the following commands, > > > > > > On my backuppc server: > > > > /cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy > > / > > > > > > On my host server: > > > > ///cat id_rsa.pub_copy >> /root/.ssh/authorized_keys/ > > > > / > > / > > I don't see any copy command from the backuppc server to the client here? > > Why dont you use > > ssh-copy-id -i /var/lib/backuppc/.ssh/id_rsa.pub root@backuppc-client > > (execute this as root user on the backuppc server!) > > > With kind regards > > Stefan Peter > > > -- > A: Because it messes up the order in which people normally read text. > Q: Why is top-posting such a bad thing? > A: Top-posting. > Q: What is the most annoying thing in e-mail? > (See https://en.wikipedia.org/wiki/Posting_style for details) > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
It should go in the authorized_keys file, not the known_hosts file. Robert Trevellyan On Mon, Mar 12, 2018 at 11:35 AM, Marc Gilliatt wrote: > Sorry my mistake, I have now corrected myself, and I have copied the RSA > key from the bakcuppc user and pasted it in the known hosts file for my > client I would like backing up, however, I am still getting the Permission > denied (publickey,password) error when I try to ssh into my client? > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 15:04:37 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > The public key you need to copy is the one for the backuppc user, not the > root user, because it's the backuppc user that's going to be making the > connection (yes, you will add it to the root user's authorized_keys on the > client, because bacuppc will sign into the client as root). > > Your initial error report shows this: > */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub"* > > This shows that it's trying to copy the key from the correct place. On my > BPC4 install, /var/lib/backuppc is symlinked to /srv/backuppc, which is why > I asked if you were working in that folder, i.e. did you 'cd /srv/backuppc' > (or 'cd /var/lib/backuppc')? > > > On Mon, Mar 12, 2018 at 10:09 AM, Marc Gilliatt > wrote: > > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > I copied and pasted the RSA key from my backuppc server to my host I would > like backing up. I ran the following commands, > > > On my backuppc server: > > > *cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy * > > > On my host server: > > *cat id_rsa.pub_copy >> /root/.ssh/authorized_keys* > > > I then switched users to backuppc, and tried to ssh to my host, however, I > was still getting *Permission denied (publickey,password).* > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 13:56:57 > > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Are you working in /srv/backuppc? > > What is the error message? > > If necessary, copy the public key to a regular text file, send that to the > host, then log into the host and append it to the authorized_keys file. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 5:33 AM, Marc Gilliatt > wrote: > > Hi Robert, > > > I tried what you had mentioned, but sadly that didn't work. I'm still > getting permission denied when I try to copy the RSA key to my host. > -- > *From:* Robert Trevellyan > *Sent:* 08 March 2018 13:14:45 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > You can work around this using something like this: > > rsync .ssh/id_rsa.pub root@10.16.0.16:/root > ssh root@10.16.0.16 > cat id_rsa.pub >> .ssh/authorized_keys > rm id_rsa.pub > exit > > Robert Trevellyan > > On Thu, Mar 8, 2018 at 6:05 AM, Marc Gilliatt > wrote: > > So I've tried to copy my RSA key to one of my hosts whilst I was root, and > that worked fine, I could successfully ssh into that host. However, when I > switch users to backuppc, and try to copy the RS key to that same host, I > still get the same error as stated in my previous email. > > > /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub" > /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed > /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys > Permission denied (publickey,password). > $ ssh root@10.16.0.16 > > I don't know why I cant copy my RSA key to my host whilst I am the > backuppc user? > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 10:54:23 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: Can't switch users to BackupPC in terminal? > > > I tried using *su - backuppc -s /bin/sh *instead and that worked, it > allowed me to switch users to backuppc, however, when I try to copy the RSA > key to my hosts I'm getting... > > > */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub"* > */usr/bin/
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
The public key you need to copy is the one for the backuppc user, not the root user, because it's the backuppc user that's going to be making the connection (yes, you will add it to the root user's authorized_keys on the client, because bacuppc will sign into the client as root). Your initial error report shows this: */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/var/lib/backuppc/.ssh/id_rsa.pub"* This shows that it's trying to copy the key from the correct place. On my BPC4 install, /var/lib/backuppc is symlinked to /srv/backuppc, which is why I asked if you were working in that folder, i.e. did you 'cd /srv/backuppc' (or 'cd /var/lib/backuppc')? On Mon, Mar 12, 2018 at 10:09 AM, Marc Gilliatt wrote: > I'm sorry, I'm not too sure on what you mean by working in /srv/backuppc? > > > I copied and pasted the RSA key from my backuppc server to my host I would > like backing up. I ran the following commands, > > > On my backuppc server: > > > *cat /root/.ssh/id_rsa.pub > id_rsa.pub_copy * > > > On my host server: > > *cat id_rsa.pub_copy >> /root/.ssh/authorized_keys* > > > I then switched users to backuppc, and tried to ssh to my host, however, I > was still getting *Permission denied (publickey,password).* > -- > *From:* Robert Trevellyan > *Sent:* 12 March 2018 13:56:57 > > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > Are you working in /srv/backuppc? > > What is the error message? > > If necessary, copy the public key to a regular text file, send that to the > host, then log into the host and append it to the authorized_keys file. > > Robert Trevellyan > > On Mon, Mar 12, 2018 at 5:33 AM, Marc Gilliatt > wrote: > > Hi Robert, > > > I tried what you had mentioned, but sadly that didn't work. I'm still > getting permission denied when I try to copy the RSA key to my host. > -- > *From:* Robert Trevellyan > *Sent:* 08 March 2018 13:14:45 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > You can work around this using something like this: > > rsync .ssh/id_rsa.pub root@10.16.0.16:/root > ssh root@10.16.0.16 > cat id_rsa.pub >> .ssh/authorized_keys > rm id_rsa.pub > exit > > Robert Trevellyan > > On Thu, Mar 8, 2018 at 6:05 AM, Marc Gilliatt > wrote: > > So I've tried to copy my RSA key to one of my hosts whilst I was root, and > that worked fine, I could successfully ssh into that host. However, when I > switch users to backuppc, and try to copy the RS key to that same host, I > still get the same error as stated in my previous email. > > > /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub" > /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed > /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys > Permission denied (publickey,password). > $ ssh root@10.16.0.16 > > I don't know why I cant copy my RSA key to my host whilst I am the > backuppc user? > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 10:54:23 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: Can't switch users to BackupPC in terminal? > > > I tried using *su - backuppc -s /bin/sh *instead and that worked, it > allowed me to switch users to backuppc, however, when I try to copy the RSA > key to my hosts I'm getting... > > > */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub"* > */usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed* > */usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys* > *Permission denied (publickey,password).* > > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 09:00:52 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* [BackupPC-users] Can't switch users to BackupPC in terminal? > > > I've installed BackupPC 4 and I cant switch users to backuppc to add hosts? > > > I've tried... > > > su - backuppc > > > but it just keeps going back to the terminal, it never switches users. > > > I'v
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
Are you working in /srv/backuppc? What is the error message? If necessary, copy the public key to a regular text file, send that to the host, then log into the host and append it to the authorized_keys file. Robert Trevellyan On Mon, Mar 12, 2018 at 5:33 AM, Marc Gilliatt wrote: > Hi Robert, > > > I tried what you had mentioned, but sadly that didn't work. I'm still > getting permission denied when I try to copy the RSA key to my host. > ------ > *From:* Robert Trevellyan > *Sent:* 08 March 2018 13:14:45 > *To:* General list for user discussion, questions and support > *Subject:* Re: [BackupPC-users] Can't switch users to BackupPC in > terminal? > > You can work around this using something like this: > > rsync .ssh/id_rsa.pub root@10.16.0.16:/root > ssh root@10.16.0.16 > cat id_rsa.pub >> .ssh/authorized_keys > rm id_rsa.pub > exit > > Robert Trevellyan > > On Thu, Mar 8, 2018 at 6:05 AM, Marc Gilliatt > wrote: > > So I've tried to copy my RSA key to one of my hosts whilst I was root, and > that worked fine, I could successfully ssh into that host. However, when I > switch users to backuppc, and try to copy the RS key to that same host, I > still get the same error as stated in my previous email. > > > /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub" > /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed > /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys > Permission denied (publickey,password). > $ ssh root@10.16.0.16 > > I don't know why I cant copy my RSA key to my host whilst I am the > backuppc user? > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 10:54:23 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: Can't switch users to BackupPC in terminal? > > > I tried using *su - backuppc -s /bin/sh *instead and that worked, it > allowed me to switch users to backuppc, however, when I try to copy the RSA > key to my hosts I'm getting... > > > */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub"* > */usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed* > */usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys* > *Permission denied (publickey,password).* > > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 09:00:52 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* [BackupPC-users] Can't switch users to BackupPC in terminal? > > > I've installed BackupPC 4 and I cant switch users to backuppc to add hosts? > > > I've tried... > > > su - backuppc > > > but it just keeps going back to the terminal, it never switches users. > > > I've tried adding hosts via the GUI, and for some reason, the backups are > failing giving the same error message... > > 2018-03-05 09:00:01 Can't find host ukat2 > <http://10.16.0.163/BackupPC_Admin?host=ukat2> via NS and netbios > 2018-03-05 09:00:01 can't ping (client = ukat2 > <http://10.16.0.163/BackupPC_Admin?host=ukat2>); exiting > > Not sure what I'm doing wrong? > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Can't switch users to BackupPC in terminal?
You can work around this using something like this: rsync .ssh/id_rsa.pub root@10.16.0.16:/root ssh root@10.16.0.16 cat id_rsa.pub >> .ssh/authorized_keys rm id_rsa.pub exit Robert Trevellyan On Thu, Mar 8, 2018 at 6:05 AM, Marc Gilliatt wrote: > So I've tried to copy my RSA key to one of my hosts whilst I was root, and > that worked fine, I could successfully ssh into that host. However, when I > switch users to backuppc, and try to copy the RS key to that same host, I > still get the same error as stated in my previous email. > > > /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub" > /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed > /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys > Permission denied (publickey,password). > $ ssh root@10.16.0.16 > > I don't know why I cant copy my RSA key to my host whilst I am the > backuppc user? > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 10:54:23 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* Re: Can't switch users to BackupPC in terminal? > > > I tried using *su - backuppc -s /bin/sh *instead and that worked, it > allowed me to switch users to backuppc, however, when I try to copy the RSA > key to my hosts I'm getting... > > > */usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: > "/var/lib/backuppc/.ssh/id_rsa.pub"* > */usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to > filter out any that are already installed* > */usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are > prompted now it is to install the new keys* > *Permission denied (publickey,password).* > > > -- > *From:* Marc Gilliatt > *Sent:* 05 March 2018 09:00:52 > *To:* backuppc-users@lists.sourceforge.net > *Subject:* [BackupPC-users] Can't switch users to BackupPC in terminal? > > > I've installed BackupPC 4 and I cant switch users to backuppc to add hosts? > > > I've tried... > > > su - backuppc > > > but it just keeps going back to the terminal, it never switches users. > > > I've tried adding hosts via the GUI, and for some reason, the backups are > failing giving the same error message... > > 2018-03-05 09:00:01 Can't find host ukat2 > <http://10.16.0.163/BackupPC_Admin?host=ukat2> via NS and netbios > 2018-03-05 09:00:01 can't ping (client = ukat2 > <http://10.16.0.163/BackupPC_Admin?host=ukat2>); exiting > > Not sure what I'm doing wrong? > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] ssh from an account without a shell
It looks like the initial connection failed. Are you able to manually SSH to the client as the backuppc user without entering a password, i.e. is key-based authentication working correctly? Robert Trevellyan On Wed, Feb 7, 2018 at 10:10 AM, Philip Parsons (Velindre - Medical Physics) wrote: > Hi Robert, > > > > Thanks for that. > > > > I’ve just tried it and (I should have already guessed this!), it gave the > same error message as BackupPC reported J > > > > Any ideas what I may be doing wrong? > > > > Thanks, > > Phil > > > > *From:* Robert Trevellyan [mailto:robert.trevell...@gmail.com] > *Sent:* 07 February 2018 14:33 > *To:* General list for user discussion, questions and support < > backuppc-users@lists.sourceforge.net> > *Subject:* Re: [BackupPC-users] ssh from an account without a shell > > > > To run a shell as the backuppc user, do something like this: > su -s /bin/bash backuppc > > > Robert Trevellyan > > > > On Wed, Feb 7, 2018 at 8:22 AM, Philip Parsons (Velindre - Medical > Physics) wrote: > > Hi all, > > > > I’ve tried looking in a couple of places, but I can’t find the answer to > this. > > > > I have essentially used this webpage to install and configure backuppc > 4.1.5 on Ubuntu 16.04 LTS - https://github.com/backuppc/ba > ckuppc/wiki/Installing-BackupPC-4-from-git-on-Ubuntu-Xenial-16.04-LTS > > > > I then tried to create an rsync backup to a SUSE server using > https://www.howtoforge.com/linux_backuppc_p4 and various other sources. > > > > When I try to run the backup it fails with ‘No files dumped for share > /data/rtgrid/’ > > > > When I run the line from the XferLOG: > > /usr/local/bin/rsync_bpc --bpc-top-dir /data/backuppc --bpc-host-name > rtgridserver --bpc-share-name /data/rtgrid/ --bpc-bkup-num 0 > --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 > --bpc-bkup-inode0 2 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -l\ > root --rsync-path=/usr/bin/sudo\ /usr/bin/rsync --super --recursive > --protect-args --numeric-ids --perms --owner --group -D --times --links > --hard-links --delete --delete-excluded --one-file-system --partial > --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum > --timeout=72000 --exclude=var/experiment --exclude=junkdrawer > --exclude=scratchpad --exclude=tmp xxx.xxx.xxx.xxx:/data/rtgrid/ > > > > As root, it runs. > > > > However, I get the error message (tagged at the bottom for readability) > when backuppc runs the job. I’m probably making things more confusing than > they should be, but I think the install webpage creates a shell-less > service backuppc user. I then can’t su to backuppc to run the line to test > it. > > > > Has anyone else used this kind of account on backuppc? Can anyone suggest > a fix for this? > > > > Thanks in advance, > > Philip Parsons > > > > > > > > Contents of file /data/backuppc/pc/rtgridserver/XferLOG.bad.z, modified > 2018-02-07 06:00:06 > > XferLOG file /data/backuppc/pc/rtgridserver/XferLOG.0.z created 2018-02-07 > 06:00:00 > > Backup prep: type = full, case = 1, inPlace = 1, doDuplicate = 0, newBkupNum > = 0, newBkupIdx = 0, lastBkupNum = , lastBkupIdx = (FillCycle = 0, noFillCnt > = ) > > Running: /usr/local/bin/rsync_bpc --bpc-top-dir /data/backuppc > --bpc-host-name rtgridserver --bpc-share-name /data/rtgrid/ --bpc-bkup-num 0 > --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 > --bpc-bkup-inode0 2 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -l\ > root --rsync-path=/usr/bin/sudo\ /usr/bin/rsync --super --recursive > --protect-args --numeric-ids --perms --owner --group -D --times --links > --hard-links --delete --delete-excluded --one-file-system --partial > --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum > --timeout=72000 --exclude=var/experiment --exclude=junkdrawer > --exclude=scratchpad --exclude=tmp xxx.xxx.xxx.xxx:/data/rtgrid/ / > > full backup started for directory /data/rtgrid/ > > Xfer PIDs are now 111286 > > This is the rsync child about to exec /usr/local/bin/rsync_bpc > > Permission denied (publickey,keyboard-interactive). > > rsync_bpc: connection unexpectedly closed (0 bytes received so far) [Receiver] > > Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 > sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 2 inode > > rsync error: unexplained error (code 255) at io.c(629) [Receiver=3.0.9.11] > > rsync_bpc exited with fatal status 255 (65280) (rsync error: unexplained > error (code 255) at io.c(629) [Receiver=3.0.9.11]) > > Xfer PIDs ar
Re: [BackupPC-users] ssh from an account without a shell
To run a shell as the backuppc user, do something like this: su -s /bin/bash backuppc Robert Trevellyan On Wed, Feb 7, 2018 at 8:22 AM, Philip Parsons (Velindre - Medical Physics) wrote: > Hi all, > > > > I’ve tried looking in a couple of places, but I can’t find the answer to > this. > > > > I have essentially used this webpage to install and configure backuppc > 4.1.5 on Ubuntu 16.04 LTS - https://github.com/backuppc/ > backuppc/wiki/Installing-BackupPC-4-from-git-on-Ubuntu-Xenial-16.04-LTS > > > > I then tried to create an rsync backup to a SUSE server using > https://www.howtoforge.com/linux_backuppc_p4 and various other sources. > > > > When I try to run the backup it fails with ‘No files dumped for share > /data/rtgrid/’ > > > > When I run the line from the XferLOG: > > /usr/local/bin/rsync_bpc --bpc-top-dir /data/backuppc --bpc-host-name > rtgridserver --bpc-share-name /data/rtgrid/ --bpc-bkup-num 0 > --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 > --bpc-bkup-inode0 2 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -l\ > root --rsync-path=/usr/bin/sudo\ /usr/bin/rsync --super --recursive > --protect-args --numeric-ids --perms --owner --group -D --times --links > --hard-links --delete --delete-excluded --one-file-system --partial > --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum > --timeout=72000 --exclude=var/experiment --exclude=junkdrawer > --exclude=scratchpad --exclude=tmp xxx.xxx.xxx.xxx:/data/rtgrid/ > > > > As root, it runs. > > > > However, I get the error message (tagged at the bottom for readability) > when backuppc runs the job. I’m probably making things more confusing than > they should be, but I think the install webpage creates a shell-less > service backuppc user. I then can’t su to backuppc to run the line to test > it. > > > > Has anyone else used this kind of account on backuppc? Can anyone suggest > a fix for this? > > > > Thanks in advance, > > Philip Parsons > > > > > > > > Contents of file /data/backuppc/pc/rtgridserver/XferLOG.bad.z, modified > 2018-02-07 06:00:06 > > XferLOG file /data/backuppc/pc/rtgridserver/XferLOG.0.z created 2018-02-07 > 06:00:00 > > Backup prep: type = full, case = 1, inPlace = 1, doDuplicate = 0, newBkupNum > = 0, newBkupIdx = 0, lastBkupNum = , lastBkupIdx = (FillCycle = 0, noFillCnt > = ) > > Running: /usr/local/bin/rsync_bpc --bpc-top-dir /data/backuppc > --bpc-host-name rtgridserver --bpc-share-name /data/rtgrid/ --bpc-bkup-num 0 > --bpc-bkup-comp 3 --bpc-bkup-prevnum -1 --bpc-bkup-prevcomp -1 > --bpc-bkup-inode0 2 --bpc-attrib-new --bpc-log-level 1 -e /usr/bin/ssh\ -l\ > root --rsync-path=/usr/bin/sudo\ /usr/bin/rsync --super --recursive > --protect-args --numeric-ids --perms --owner --group -D --times --links > --hard-links --delete --delete-excluded --one-file-system --partial > --log-format=log:\ %o\ %i\ %B\ %8U,%8G\ %9l\ %f%L --stats --checksum > --timeout=72000 --exclude=var/experiment --exclude=junkdrawer > --exclude=scratchpad --exclude=tmp xxx.xxx.xxx.xxx:/data/rtgrid/ / > > full backup started for directory /data/rtgrid/ > > Xfer PIDs are now 111286 > > This is the rsync child about to exec /usr/local/bin/rsync_bpc > > Permission denied (publickey,keyboard-interactive). > > rsync_bpc: connection unexpectedly closed (0 bytes received so far) [Receiver] > > Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 > sizeTotal, 0 filesNew, 0 sizeNew, 0 sizeNewComp, 2 inode > > rsync error: unexplained error (code 255) at io.c(629) [Receiver=3.0.9.11] > > rsync_bpc exited with fatal status 255 (65280) (rsync error: unexplained > error (code 255) at io.c(629) [Receiver=3.0.9.11]) > > Xfer PIDs are now > > Got fatal error during xfer (No files dumped for share /data/rtgrid/) > > Backup aborted (No files dumped for share /data/rtgrid/) > > BackupFailCleanup: nFilesTotal = 0, type = full, BackupCase = 1, inPlace = 1, > lastBkupNum = > > Removing empty backup #0 > > Xfer PIDs are now 111317 > > BackupPC_backupDelete: removing #0 > > BackupPC_backupDelete: No prior backup for merge > > Xfer PIDs are now 111317,111318 > > BackupPC_refCountUpdate: host rtgridserver got 0 errors (took 0 secs) > > Xfer PIDs are now 111317 > > Xfer PIDs are now > > Finished BackupPC_backupDelete, status = 0 (running time: 0 sec) > > Xfer PIDs are now > > Running BackupPC_refCountUpdate -h rtgridserver -f on rtgridserver > > Xfer PIDs are now 111319 > > BackupPC_refCountUpdate: host rtgridserver got 0 errors (took 0 secs) > > Xfer PIDs are now > >
