Re: [DNG] What not to back up
Bob Proulx via Dng said on Sun, 28 Nov 2021 10:57:36 -0700 >Mike Tubby wrote: >> ... but if you run a nameserver you may well need: >> >> /var/cache/bind >> >> as that's where your zonefiles are ;-) > >Sorry. No. I am curious what led you to that conclusion? > >By default in the Debian packaged configuration only the cached zone >files downloaded on secondaries are located there. (The upstream BIND >does not specify a default location. This is a distro package default >location.) Since it is a package default the local admin may also >change it to any other location they wish. But /var/cache is as good >of a location as any for cached files. > >And therefore by all means delete that directory any time you feel >like doing so and recreate it empty. Now that I have this new information, I retract my particular insult for BIND. I still don't like it for various reasons, but fair's fair; at least they didn't put config in a cache directory. SteveT Steve Litt Spring 2021 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Mike Tubby wrote: > ... but if you run a nameserver you may well need: > > /var/cache/bind > > as that's where your zonefiles are ;-) Sorry. No. I am curious what led you to that conclusion? By default in the Debian packaged configuration only the cached zone files downloaded on secondaries are located there. (The upstream BIND does not specify a default location. This is a distro package default location.) Since it is a package default the local admin may also change it to any other location they wish. But /var/cache is as good of a location as any for cached files. And therefore by all means delete that directory any time you feel like doing so and recreate it empty. It's not important and does not require being backed up. It only needs to exist so the named has a location to store and cache files (that have a TTL too) downloaded from the primary. If that directory is empty then upon start the BIND named will request a fresh download of all of the zones it is configured for as a secondary nameserver and will cache them in that directory again. If the named is not configured as a secondary then that directory will be empty of zone files. For DNS primaries one specifies the source zone file using the named.conf "file" directive. Put that file anywhere you wish to put it. But putting that in /var/cache/bind would be a very poor choice in my opinion. Example of actually doing this. file "/etc/bind/db.proulx.com"; Since this is an option that must be configured when setting up a primary zone then you can put those source zone files anywhere you decide is the place to keep the source of them. I highly recommend etckeeper for all of /etc and therefore I prefer to keep source there where etckeeper can track them. Bob signature.asc Description: PGP signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Or, tell bind to place the zone files where they originally were, in /etc/bind/zones or something. The change was made about 10 years ago as a "security feature" and is mainly used for running bind in a jail, so if it gets hacked, they can't mess up the rest of the server. I remember when Debian went that way and it confused me quite a bit. Of course, if you have a dedicated server only for BIND, that reason goes away. So, simply edit /etc/bind/* and change /var/lib/bind to whatever you want. For the most part, I just store them in /etc/bind/SEC or /etc/bind/ZONES or something. BIND doesn't care; it is the distro people doing that. Rod On 11/26/21 7:07 AM, Mike Tubby wrote: > > > On 24/11/2021 10:08, Olaf Meeuwissen via Dng wrote: >> Hi Hendrik, >> >> Hendrik Boom writes: >> >>> I'm setting up a new backup script that will do it all piecemeal so >>> that if a part of it fails, it can be retried without having to start >>> *everythng* over from scratch. >>> >>> Which top-level filesystems should *not* be backed up. >>> >>> To start with, I presumably shouldn't back up >>> >>> /proc >>> /tmp >>> /dev (cause I'm using some version of *udev) >>> /mnt >> ACK. >> >>> and I certainly should back up /var, /usr. /root, /bin, >>> /boot, /etc, /home, /lib, /lib64, /sbin >> I wouldn't bother with /var/cache and /var/log but you're talking >> top-level ;-) > > ... but if you run a nameserver you may well need: > > /var/cache/bind > > as that's where your zonefiles are ;-) > > >> /boot is managed by installing kernel images and grub (using settings in >> /etc/grub) so isn't all that important to include. At least on amd64. >> >>> But what about >>> >>> /run >>> /srv >>> /sys >>> ? >> Both /run and /sys are tmpfs file systems. Not worth backing up. > > > However some admins put services in: > > /srv > > and some third-party suppliers of software place it in: > > /opt > > for example Sophos anti-virus. > >> Basically, you should only care about a subset of what lives below the >> mount points listed by >> >> df | grep ^/ | awk '{print $6}' >> >> and make sure your backup command doesn't cross file system boundaries. >> That should automatically exclude things like /dev, /proc, /run, /sys >> and may (or may not) exclude /tmp (depending on installation choices). >> As /mnt is meant for temporary mounts, that should be excluded too. >> >>> What are those even used for? >> I would have pointed you to the FHS but as Lars pointed out already `man >> 7 hier` will tell. >> >> Of course, if you don't use things like /srv and /opt, there's not much >> of a cost to backing up the empty directories :-) >> >> Hope this helps, >> -- >> Olaf Meeuwissen, LPIC-2 FSF Associate Member since 2004-01-27 >> GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 >> Support Free Software https://my.fsf.org/donate >> Join the Free Software Foundation https://my.fsf.org/join >> ___ >> Dng mailing list >> Dng@lists.dyne.org >> https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng > > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng -- Rod Rodolico Daily Data, Inc. POB 140465 Dallas TX 75214-0465 US https://dailydata.net 214.827.2170 ext 100 ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Mike Tubby said on Fri, 26 Nov 2021 13:07:36 + > >... but if you run a nameserver you may well need: > > /var/cache/bind > >as that's where your zonefiles are ;-) Thanks for reminding me again one of the reasons I don't use bind. Who in their right mind would put zone files in a cache directory? Sounds like something Poetterpinhead would do. I'm an unbound/nsd guy myself. Before that I was a djbdns guy. Both put config in config directories. SteveT Steve Litt Spring 2021 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
On 24/11/2021 10:08, Olaf Meeuwissen via Dng wrote: Hi Hendrik, Hendrik Boom writes: I'm setting up a new backup script that will do it all piecemeal so that if a part of it fails, it can be retried without having to start *everythng* over from scratch. Which top-level filesystems should *not* be backed up. To start with, I presumably shouldn't back up /proc /tmp /dev (cause I'm using some version of *udev) /mnt ACK. and I certainly should back up /var, /usr. /root, /bin, /boot, /etc, /home, /lib, /lib64, /sbin I wouldn't bother with /var/cache and /var/log but you're talking top-level ;-) ... but if you run a nameserver you may well need: /var/cache/bind as that's where your zonefiles are ;-) /boot is managed by installing kernel images and grub (using settings in /etc/grub) so isn't all that important to include. At least on amd64. But what about /run /srv /sys ? Both /run and /sys are tmpfs file systems. Not worth backing up. However some admins put services in: /srv and some third-party suppliers of software place it in: /opt for example Sophos anti-virus. Basically, you should only care about a subset of what lives below the mount points listed by df | grep ^/ | awk '{print $6}' and make sure your backup command doesn't cross file system boundaries. That should automatically exclude things like /dev, /proc, /run, /sys and may (or may not) exclude /tmp (depending on installation choices). As /mnt is meant for temporary mounts, that should be excluded too. What are those even used for? I would have pointed you to the FHS but as Lars pointed out already `man 7 hier` will tell. Of course, if you don't use things like /srv and /opt, there's not much of a cost to backing up the empty directories :-) Hope this helps, -- Olaf Meeuwissen, LPIC-2FSF Associate Member since 2004-01-27 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 Support Free Softwarehttps://my.fsf.org/donate Join the Free Software Foundation https://my.fsf.org/join ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Hi, Steve Litt writes: > Dr. Nikolaus Klepp via Dng said on Tue, 23 Nov 2021 20:43:28 +0100 > >>Anno domini 2021 Tue, 23 Nov 14:27:56 -0500 >> Hendrik Boom scripsit: >>> I'm setting up a new backup script that will do it all piecemeal so >>> that if a part of it fails, it can be retried without having to >>> start *everythng* over from scratch. >>> >>> Which top-level filesystems should *not* be backed up. >> >> >>Question is: What do you want the backup for? Recover from a failed >>disk in 5 minutes or "just" all your settings and user directories? I >>for myself do not bother to save the OS, a list of all manually >>installed packages is good enough for me. >> >>Nik > > I'm the same as Nik. If I can buy it again, or install it again, it's > not a tragedy if I lose it. For this reason I don't back up /usr. By the same reasoning, you can exclude /bin, /lib, /lib64 and /sbin. However, I would back up /usr/local because that may be a real pain to reconstruct. > > The majority of files in /home/yourname are useless. /home/yourname is > a mishmash of stuff you created, settings you use, and useless crap > like cache. It's huge and ugly. For that reason I create other top > level directories to hold stuff I created myself. I create dedicated directories below $HOME, version control them with git and push to an external location. Gives me backups + versioning! > > > Nevertheless, it really is necessary to back up /home, although > everything should be done to make sure none of what you back up is > cache: > > == > [slitt@mydesk ~]$ find .cache | wc -l > 82571 > [slitt@mydesk ~]$ du -hs .cache > 2.1G .cache > [slitt@mydesk ~]$ find . | grep cache | wc -l > find: ‘./mail/Maildir/lost+found’: Permission denied > 173948 > [slitt@mydesk ~]$ > == > > Really? Depending on what software you use, you may have missed a swat of cache. I'd look for find $HOME/.[^.]* -iname '*cache' -type d # The above assumes bash for the .[^.]* shell glob. At the office I (have to) use M$ Teams and that caches boatloads of stuff elsewhere. Firefox also creates per site caches. > Then there's ~/Downloads. The way I see it, if you need things in the > download directory enough to back them up, those files should have been > moved somewhere else. That'd be ${XDG_DOWNLOAD_DIR:-Downloads} for folks that customize. # Settings in ${XDG_CONFIG_HOME:-$HOME/.config}/user-dirs.dirs. > I back up /home minus .cache, but I segregate that backup, and when I > reinstall, instead of restoring to /home, I restore it to > /scatch/oldhome, and manually transfer things as necessary. I also do something like that for my $HOME. Now I still have an $HOME/__ATTIC__/__ATTIC__/__ATTIC__/__ATTIC__/ that needs cleaning out ;-O > In my opinion, here are some things that are absolutely essential to > back up: > > * The /etc tree > * The output of the mount command (yeah, I know /etc/fstab, but still) > * The output of the command telling all the packages that were > installed manually. As mentioned in another follow up, you may want a *full* list of packages installed (with their versions) if only to help trouble shooting issues after reinstalling the list of previously manually installed packages. Even if you installed all 50,000+ Devuan packages, that'd be peanuts compared to the size of the rest of your backup. > * The UMENU2 menu structure. I have no such thing as far as I am aware of. Don't even know what UMENU2 is :-? Ah, http://troubleshooters.com/projects/umenu2/ explains. Don't have it, then. For my needs, dmenu and *sh-completion suffice. > * All data you created, and I hope it's *not* in /home. > > Like Nik says, if your goal is to get it back up in 5 minutes, your > best bet is to back up the entire system, as well as the mbr or > whatever you call the UEFI equivalent (both copies). But if your > intent is just to stay in business after losing a disk, I think a > data-only backup is superior. Hope this helps, -- Olaf Meeuwissen, LPIC-2FSF Associate Member since 2004-01-27 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 Support Free Softwarehttps://my.fsf.org/donate Join the Free Software Foundation https://my.fsf.org/join ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Hi Hendrik, Hendrik Boom writes: > I'm setting up a new backup script that will do it all piecemeal so > that if a part of it fails, it can be retried without having to start > *everythng* over from scratch. > > Which top-level filesystems should *not* be backed up. > > To start with, I presumably shouldn't back up > > /proc > /tmp > /dev (cause I'm using some version of *udev) > /mnt ACK. > and I certainly should back up /var, /usr. /root, /bin, > /boot, /etc, /home, /lib, /lib64, /sbin I wouldn't bother with /var/cache and /var/log but you're talking top-level ;-) /boot is managed by installing kernel images and grub (using settings in /etc/grub) so isn't all that important to include. At least on amd64. > But what about > > /run > /srv > /sys > ? Both /run and /sys are tmpfs file systems. Not worth backing up. Basically, you should only care about a subset of what lives below the mount points listed by df | grep ^/ | awk '{print $6}' and make sure your backup command doesn't cross file system boundaries. That should automatically exclude things like /dev, /proc, /run, /sys and may (or may not) exclude /tmp (depending on installation choices). As /mnt is meant for temporary mounts, that should be excluded too. > What are those even used for? I would have pointed you to the FHS but as Lars pointed out already `man 7 hier` will tell. Of course, if you don't use things like /srv and /opt, there's not much of a cost to backing up the empty directories :-) Hope this helps, -- Olaf Meeuwissen, LPIC-2FSF Associate Member since 2004-01-27 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 Support Free Softwarehttps://my.fsf.org/donate Join the Free Software Foundation https://my.fsf.org/join ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Hi, Dr. Nikolaus Klepp via Dng writes: > Anno domini 2021 Tue, 23 Nov 21:39:07 +0200 > Lars Noodén via Dng scripsit: >> On 11/23/21 21:27, Hendrik Boom wrote: >> > I'm setting up a new backup script that will do it all piecemeal so >> > that if a part of it fails, it can be retried without having to start >> > *everythng* over from scratch. >> [snip] >> >> It depends on what you've set up. >> >> For the systems I have, I only back up the configuration files in /etc/ >> plus the output of >> dpkg --get-selections >> The restoration plan is to do a fresh installation and restore with >> dpkg --set-selections > > I prefer "apt-mark showmanual" :) That gets my vote as well but ... > $ dpkg --get-selections|wc -l > 3474 > $ apt-mark showmanual|wc -l > 904 ... changing dependencies being what they are, one might have a few dependencies lingering around that would not be covered by apt-mark's output. In case one unwittingly relied on those being installed, the output of `dpkg --get-selections` or `dpkg-query -W` might be a better option. Note that the latter even includes version info. I would include the output of all three in a backup. It's not a lot of info, just a couple of kb, but may save the day in case restoring does not quite get you where you want to be. Hope this helps, -- Olaf Meeuwissen, LPIC-2FSF Associate Member since 2004-01-27 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 Support Free Softwarehttps://my.fsf.org/donate Join the Free Software Foundation https://my.fsf.org/join ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Harald Arnesen via Dng said on Tue, 23 Nov 2021 21:56:43 +0100 >Steve Litt [23/11/2021 21.48]: > >> >> The majority of files in /home/yourname are useless. /home/yourname >> is a mishmash of stuff you created, settings you use, and useless >> crap > >> like cache. It's huge and ugly. For that reason I create other top >> level directories to hold stuff I created myself. >> > >Really? You don't backup your settings? I would think those were some >of the most important things. Read my entire post. SteveT Steve Litt Spring 2021 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Steve Litt [23/11/2021 21.48]: The majority of files in /home/yourname are useless. /home/yourname is a mishmash of stuff you created, settings you use, and useless crap like cache. It's huge and ugly. For that reason I create other top level directories to hold stuff I created myself. Really? You don't backup your settings? I would think those were some of the most important things. -- Hilsen Harald ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Dr. Nikolaus Klepp via Dng said on Tue, 23 Nov 2021 20:43:28 +0100 >Anno domini 2021 Tue, 23 Nov 14:27:56 -0500 > Hendrik Boom scripsit: >> I'm setting up a new backup script that will do it all piecemeal so >> that if a part of it fails, it can be retried without having to >> start *everythng* over from scratch. >> >> Which top-level filesystems should *not* be backed up. > > >Question is: What do you want the backup for? Recover from a failed >disk in 5 minutes or "just" all your settings and user directories? I >for myself do not bother to save the OS, a list of all manually >installed packages is good enough for me. > >Nik I'm the same as Nik. If I can buy it again, or install it again, it's not a tragedy if I lose it. For this reason I don't back up /usr. The majority of files in /home/yourname are useless. /home/yourname is a mishmash of stuff you created, settings you use, and useless crap like cache. It's huge and ugly. For that reason I create other top level directories to hold stuff I created myself. Nevertheless, it really is necessary to back up /home, although everything should be done to make sure none of what you back up is cache: == [slitt@mydesk ~]$ find .cache | wc -l 82571 [slitt@mydesk ~]$ du -hs .cache 2.1G.cache [slitt@mydesk ~]$ find . | grep cache | wc -l find: ‘./mail/Maildir/lost+found’: Permission denied 173948 [slitt@mydesk ~]$ == Really? Then there's ~/Downloads. The way I see it, if you need things in the download directory enough to back them up, those files should have been moved somewhere else. I back up /home minus .cache, but I segregate that backup, and when I reinstall, instead of restoring to /home, I restore it to /scatch/oldhome, and manually transfer things as necessary. In my opinion, here are some things that are absolutely essential to back up: * The /etc tree * The output of the mount command (yeah, I know /etc/fstab, but still) * The output of the command telling all the packages that were installed manually. * The UMENU2 menu structure. * All data you created, and I hope it's *not* in /home. Like Nik says, if your goal is to get it back up in 5 minutes, your best bet is to back up the entire system, as well as the mbr or whatever you call the UEFI equivalent (both copies). But if your intent is just to stay in business after losing a disk, I think a data-only backup is superior. SteveT Steve Litt Spring 2021 featured book: Troubleshooting Techniques of the Successful Technologist http://www.troubleshooters.com/techniques ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Anno domini 2021 Tue, 23 Nov 21:39:07 +0200 Lars Noodén via Dng scripsit: > On 11/23/21 21:27, Hendrik Boom wrote: > > I'm setting up a new backup script that will do it all piecemeal so > > that if a part of it fails, it can be retried without having to start > > *everythng* over from scratch. > [snip] > > It depends on what you've set up. > > For the systems I have, I only back up the configuration files in /etc/ > plus the output of > dpkg --get-selections > The restoration plan is to do a fresh installation and restore with > dpkg --set-selections I prefer "apt-mark showmanual" :) $ dpkg --get-selections|wc -l 3474 $ apt-mark showmanual|wc -l 904 nik > > Then for the data, it is /var/ and /home/, with special treatment for > any live databases if needed. I don't use /srv/ or /opt/ for anything. > > > But what about > > > > /run > > /srv > > /sys > > ? > > > > What are those even used for? > > See "man 7 hier" > > """ > /run This directory contains information which describes > the system since it was booted. Once this purpose > was served by /var/run and programs may continue > to use it. > > /srv This directory contains site-specific data that is > served by this system. > > /sys This is a mount point for the sysfs filesystem, which > provides information about the kernel like /proc, but > better structured, following the formalism of kobject > infrastructure. > """ > > > > /Lars > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng > -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
Anno domini 2021 Tue, 23 Nov 14:27:56 -0500 Hendrik Boom scripsit: > I'm setting up a new backup script that will do it all piecemeal so > that if a part of it fails, it can be retried without having to start > *everythng* over from scratch. > > Which top-level filesystems should *not* be backed up. Question is: What do you want the backup for? Recover from a failed disk in 5 minutes or "just" all your settings and user directories? I for myself do not bother to save the OS, a list of all manually installed packages is good enough for me. Nik > > To start with, I presumably shouldn't back up > > /proc > /tmp > /dev (cause I'm using some version of *udev) > /mnt > > and I certainly should back up /var, /usr. /root, /bin, > /boot, /etc, /home, /lib, /lib64, /sbin > > But what about > > /run > /srv > /sys > ? > > What are those even used for? > > -- hendrik > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng > -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] What not to back up
On 11/23/21 21:27, Hendrik Boom wrote: I'm setting up a new backup script that will do it all piecemeal so that if a part of it fails, it can be retried without having to start *everythng* over from scratch. [snip] It depends on what you've set up. For the systems I have, I only back up the configuration files in /etc/ plus the output of dpkg --get-selections The restoration plan is to do a fresh installation and restore with dpkg --set-selections Then for the data, it is /var/ and /home/, with special treatment for any live databases if needed. I don't use /srv/ or /opt/ for anything. > But what about > > /run > /srv > /sys > ? > > What are those even used for? See "man 7 hier" """ /run This directory contains information which describes the system since it was booted. Once this purpose was served by /var/run and programs may continue to use it. /srv This directory contains site-specific data that is served by this system. /sys This is a mount point for the sysfs filesystem, which provides information about the kernel like /proc, but better structured, following the formalism of kobject infrastructure. """ /Lars ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng