Re: Need help speeding up a backup
I don't know why your copy is slow, but I know how to make it take less time. I think you are defeating the purpose of rsync, you are creating a new directory every time, which means it needs to copy everything every time. What I do is do an rsync -rau --delete from the directory I want to copy to the same place every time i.e. rsync -rau --delete excludedirs fromdir todir rsync is smart enough to only do copies of files that changed, your command is copying some files that never change or change very few times. Doing it to the same place makes the sync go much faster. After the sync, I then do a tar czf output-daily-todaysdate-time.tgz ./directorytosave then I do a find output-daily* and delete the older ones after so many days I also use fsarchiver to make a copy of my system partition (it does copy everything) On Fri, Jul 15, 2022 at 4:32 PM Matt Graham via PLUG-discuss < plug-discuss@lists.phxlinux.org> wrote: > On 2022-07-15 08:40, Mark Phillips via PLUG-discuss wrote: > > I am trying to back up the contents of the drives (ie /) to an > > external usb drive using rsync. It is taking a really long time. > > After 26 hours of continuous operation I have only transferred > > 138 GB out of 2+ TB > > 1G takes about 11.3 minutes? That's super slow. 1G takes about 30 > seconds on a USB2 external disk here. How is this mounted? > > > rsync --no-compress --info=progress2 -avAXEWSlHh --exclude= > > {'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys', > > '/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'} > > / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/' > > Any suggestions on how I can speed this up and not lose any data? > > You don't need -AX unless you're using ACLs and xattrs, and you > probably aren't. You probably don't want -W. -H is IIRC heavyweight > and you probably don't have many (or any) hard links on your > filesystems. The path to the backup disk looks like an automounter is > involved, that can cause dumb things like being mounted with sync. > > First suggestion, just use -av . Second suggestion, instead of trying > to back up everything, back up only the irreplaceable stuff. This is > usually /home and /etc , maybe /usr/local . When rsyncing a lot of > stuff (800G not 2T though), a significant amount of time gets spent > stat()ing every single file and dir on both source and destination. How > many files are you dealing with? /home here has 272G and 711889 inodes > according to df -i. An initial sync would take roughly 136 minutes, > subsequent syncs would take roughly 4 minutes. That's what I recall > from the last time I used rsync to backup. > > While it's written in Golang and therefore suspect, restic ( > https://restic.github.io/ ) has an interesting feature that allows you > to retain a varying number of hourly, daily, weekly, monthly, and yearly > snapshots of whatever it is you want to back up. The documentation > about how to get to a specific restore point is not great, and the > "mount restore point" functionality horks up directory permissions. > However, it's quite fast at backing up stuff, and it can use S3 buckets > and some other not-your-computer networky things instead of local disks > as backup destinations. > > -- > Crow202 Blog: http://crow202.org/wordpress > There is no Darkness in Eternity > But only Light too dim for us to see. > --- > PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org > To subscribe, unsubscribe, or to change your mail settings: > https://lists.phxlinux.org/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss
Re: Dock Replacement under Wayland
GLX-Dock is actually the older form of Cairo dock, go figure, probably broken too I'd presume. Lemme see! Sadly no docks seem to work worth a damn in wayland. Argh! FML! Wayland defeated by shite dock software. -mb On Fri, Jul 15, 2022 at 9:46 PM Stephen Partington wrote: > You try this one yet? > > https://glx-dock.org/mr_article.php?b=5&a=73 > > On Sat, Jul 16, 2022, 12:44 AM Michael Butash via PLUG-discuss < > plug-discuss@lists.phxlinux.org> wrote: > >> I've used cairo-dock forever, sort of an unforgiving bastard, but the >> best dock out there imho for linux still. I've tried a lot, but found >> cairo doesn't work under wayland that is my weekend adventure to replace >> the infernal xorg compositors worth about shite for anything 4k res or >> higher. >> >> Wayland seems so far like 1000% better than standard kwin compositing in >> the first hour(s), I just really need something like cairo-dock I've used >> for like a decade or more. >> >> Any thoughts on a replacement? Not seeing much right now, can't be this >> bad in the wayland landscape still, is it? >> >> -mb >> --- >> PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org >> To subscribe, unsubscribe, or to change your mail settings: >> https://lists.phxlinux.org/mailman/listinfo/plug-discuss > > --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss
Re: Dock Replacement under Wayland
You try this one yet? https://glx-dock.org/mr_article.php?b=5&a=73 On Sat, Jul 16, 2022, 12:44 AM Michael Butash via PLUG-discuss < plug-discuss@lists.phxlinux.org> wrote: > I've used cairo-dock forever, sort of an unforgiving bastard, but the best > dock out there imho for linux still. I've tried a lot, but found cairo > doesn't work under wayland that is my weekend adventure to replace the > infernal xorg compositors worth about shite for anything 4k res or higher. > > Wayland seems so far like 1000% better than standard kwin compositing in > the first hour(s), I just really need something like cairo-dock I've used > for like a decade or more. > > Any thoughts on a replacement? Not seeing much right now, can't be this > bad in the wayland landscape still, is it? > > -mb > --- > PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org > To subscribe, unsubscribe, or to change your mail settings: > https://lists.phxlinux.org/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss
Dock Replacement under Wayland
I've used cairo-dock forever, sort of an unforgiving bastard, but the best dock out there imho for linux still. I've tried a lot, but found cairo doesn't work under wayland that is my weekend adventure to replace the infernal xorg compositors worth about shite for anything 4k res or higher. Wayland seems so far like 1000% better than standard kwin compositing in the first hour(s), I just really need something like cairo-dock I've used for like a decade or more. Any thoughts on a replacement? Not seeing much right now, can't be this bad in the wayland landscape still, is it? -mb --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss
Re: Need help speeding up a backup
On 2022-07-15 08:40, Mark Phillips via PLUG-discuss wrote: I am trying to back up the contents of the drives (ie /) to an external usb drive using rsync. It is taking a really long time. After 26 hours of continuous operation I have only transferred 138 GB out of 2+ TB 1G takes about 11.3 minutes? That's super slow. 1G takes about 30 seconds on a USB2 external disk here. How is this mounted? rsync --no-compress --info=progress2 -avAXEWSlHh --exclude= {'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys', '/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'} / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/' Any suggestions on how I can speed this up and not lose any data? You don't need -AX unless you're using ACLs and xattrs, and you probably aren't. You probably don't want -W. -H is IIRC heavyweight and you probably don't have many (or any) hard links on your filesystems. The path to the backup disk looks like an automounter is involved, that can cause dumb things like being mounted with sync. First suggestion, just use -av . Second suggestion, instead of trying to back up everything, back up only the irreplaceable stuff. This is usually /home and /etc , maybe /usr/local . When rsyncing a lot of stuff (800G not 2T though), a significant amount of time gets spent stat()ing every single file and dir on both source and destination. How many files are you dealing with? /home here has 272G and 711889 inodes according to df -i. An initial sync would take roughly 136 minutes, subsequent syncs would take roughly 4 minutes. That's what I recall from the last time I used rsync to backup. While it's written in Golang and therefore suspect, restic ( https://restic.github.io/ ) has an interesting feature that allows you to retain a varying number of hourly, daily, weekly, monthly, and yearly snapshots of whatever it is you want to back up. The documentation about how to get to a specific restore point is not great, and the "mount restore point" functionality horks up directory permissions. However, it's quite fast at backing up stuff, and it can use S3 buckets and some other not-your-computer networky things instead of local disks as backup destinations. -- Crow202 Blog: http://crow202.org/wordpress There is no Darkness in Eternity But only Light too dim for us to see. --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss
Need help speeding up a backup
I have an Ubuntu 18.04 system with two drives in an lvm with one logical root partition. I am trying to back up the contents of the drives (ie /) to an external usb drive using rsync. It is taking a really long time. After 26 hours of continuous operation I have only transferred 138 GB out of 2+ TB, so I am looking at about 16 days to complete the transfer. My rsync command is: sudo rsync --no-compress --info=progress2 -avAXEWSlHh --exclude={'/run','/mnt','/swapfile','/boot','/dev','/proc','/sys','/run','/mnt','/media','/lost+found','/swapfile.extended','/tmp'} / '/media/mark/Seagate Portable Drive/tsunami-backups-Jul_13_17-39/' Any suggestions on how I can speed this up and not lose any data? Thanks! Mark --- PLUG-discuss mailing list - PLUG-discuss@lists.phxlinux.org To subscribe, unsubscribe, or to change your mail settings: https://lists.phxlinux.org/mailman/listinfo/plug-discuss