Re: archive files
On Fri, May 05, 2023 at 12:07:09AM -0700, David Christensen wrote: How does cpio(1) compare? How about dump(8) and restore(8)? I'd note that these (like tar) are both very old tools, and there are many more modern ones perhaps worthy of consideration too. (which; is left as an exercise for the reader) AIUI dd(1) produces identical backup/ restore results. Do you know otherwise? It does so by cloning the device underneath the filesystem, and thus, the filesystem on top (and all the 'gaps' between); so yes, using dd(1) can preserve any metadata that you care about. But the result is very unwieldy to store or access, and you have to be sure the filesystem is in a ready state to be cloned or information that has not hit the disk (in data structures in memory or buffer cache) could be lost, so it's not generally recommended as a tool to produce a file archive. -- Please do not CC me for listmail. 👱🏻 Jonathan Dowland ✎j...@debian.org 🔗 https://jmtd.net
archive files
On 5/4/23 23:46, Jeremy Ardley wrote: On 5/5/23 14:21, David Christensen wrote: If the file is a some kind of archive (e.g. tar(1)), both the data and metadata are inside the tarball and the full-circle results should be identical. Not quite. The file times are usually changed in the un-tar operation. You can usually expect to only have a valid m-time. In addition, extended attributes such as ACL are not preserved without some effort. There is a further problem in that the owner and group values may be different between systems The situation is much worse with NTFS on windows where there are alternative data streams and up to 16 timestamps per file per datastream. Thank you for the clarification. Fortunately, I use the same (user) UID's/GID's on all of my (Unix) systems, I have not needed ACL's, and simple use of tar(1) has preserved the metadata that I do care about (UID, GID, mode bits, mtime, and filename). How does cpio(1) compare? How about dump(8) and restore(8)? AIUI dd(1) produces identical backup/ restore results. Do you know otherwise? David
Re: Are /var/cache/apt/archive files essential?
Maurits van Rees <[EMAIL PROTECTED]> wrote: From the apt-get manual page: clean autoclean So call 'apt-get autoclean' to remove most of your packages or 'apt-get clean' to remove them all. It doesn't hurt. You just need to fetch those files again when you have to reinstall for some reason. Note though that having an older version of a package lying around in the apt cache can be useful when you discover a bug in the new version and want to revert to the old. To have more control, cobble together some shell commands. For example see this posting by me: http://lists.debian.org/debian-user/2004/10/msg01671.html I have never been happy with the 'clean' and 'autoclean' options for apt-get. I don't want to remove the files for packages that I currently have installed. I just want to remove the old stuff. I have yet to find anything that will do this for me. That is why my /var partition also fills up until something stops working and I manually go through the directory and start deleting big files. Well, I recently got tired of that (it is tedius and inefficient) so I wrote a python script to automate a lot of it. (If I had had Maurits' scripts I probably would have been hapy with that, but now I have my own.) The scripts looks through /var/cache/apt/archives and the output of 'dpkg -l'. It then displays some statistics about the archive, i.e. Total installed packages Total uninstalled packages in archive Total packages in archive Total files in archive Packages in archive w/dups Files in archive in dup pkgs The following prompts will then be presented (taken from the program's help screen): Delete uninstalled packages from archive (y/N/p/q)? y = Delete all uninstalled package files from the archives without prompting. N = Do not delete any package files based on installation status. p = Prompt to keep each uninstalled package file in the archive. q = Quit program without deleting anything. If prompting was requested: Keep: filename.deb (Y/n/q) ? Y = Keep the package file in the archives (do not delete). n = Delete the package file from the archives. q = Quit program without deleting the package file. For each package with multiple files in the archive a list of each file will be presented. The list is presented in the system sort order and an asterisk (*) is displayed before the file with the most recent date. For the last package shown the following prompt will be presented. Keep only filename.deb (y/N/d/q) ? y = Keep the listed package file (deleting all others for the specified package). N = Prompt to keep each package file. d = Keep only the most recent file (as indicated with an asterisk). q = Quit program without deleting the package file. If prompting for package deletion was requested: Keep: filename.deb (Y/n/q) ? Y = Keep the package in the archives (do not delete). n = Delete the package from the archives. q = Quit program without deleting the package. NOTE: All responses are case insensitive and require only a single character. Defaults are shown in uppercase. Taking all defaults WILL NOT DELETE ANYTHING! --- So far, everything seems to work as expected. I have used this on both of my boxes, one is running Sarge (w/ Python 2.3) and the other is running Woody (w/ Python 2.1). There is extra code in the script to handle the older 'ls' from Woody which does not have an option to display the date in a sortable fashion, so this should work on older systems, as well as newer ones. Just the option to remove packages not listed as installed by dpkg removed 117 files on my Sarge box. All in all I have used it to free up nearly 300MB of space (the archives now take up about 492MB. Since I have 1GB in /var that is almost a third of my available space freed up. If anyone is interested in trying this out, drop me an e-mail and I will send it out to you. I would appreciate any comments, or feedback. -- Marc Shapiro [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Are archive files essential? - Thanks
On Fri, Jun 03, 2005 at 10:18:42PM +0200, Maurits van Rees wrote: > On Fri, Jun 03, 2005 at 03:44:46PM -0400, Thomas H. George wrote: > > I note that there are multiple entries for most of the packages in the > > archive with dates going back two or three years. Perhaps it would have > > been better to weed out the older entries. If so, is there a simple way > > to do this? > > From the apt-get manual page: > > >clean > > clean clears out the local repository of retrieved package files. It > removes everything but the lock file from /var/cache/apt/archives/ and > /var/cache/apt/archives/partial/. When APT is used as a dselect(8) > method, clean is run automatically. Those who do not use dselect will > likely want to run apt-get clean from time to time to free up disk > space. > >autoclean > > Like clean, autoclean clears out the local repository of retrieved > package files. The differ- ence is that it only removes package files > that can no longer be downloaded, and are largely useless. This > allows a cache to be maintained over a long period without it growing > out of control. The configuration option APT::Clean-Installed will > prevent installed packages from being erased if it is set to off. > > > So call 'apt-get autoclean' to remove most of your packages or > 'apt-get clean' to remove them all. It doesn't hurt. You just need to > fetch those files again when you have to reinstall for some reason. > > Note though that having an older version of a package lying around in > the apt cache can be useful when you discover a bug in the new version > and want to revert to the old. > > To have more control, cobble together some shell commands. For example > see this posting by me: > > http://lists.debian.org/debian-user/2004/10/msg01671.html > > At the end that post boils down to the following shell line that > prints all packages that have a more than two versions in that > directory. > > ls /var/cache/apt/archives | sort -g | uniq -c -t "_" -d -W 1 | grep -v -E > "^[[:blank:]]*2 " | sed "s/ *[[:digit:]*] \(.*\)$/\1/g" > > > I have another script in /usr/local/bin/cleanpackages that moves old > packages to another directory. That should then be a directory on a > partition where you have more space. Instead of moving them you can > delete them if you want. > > > #! /bin/sh > > # Move old files to another dir. Mostly meant for the apt cache. > > if test $# -eq 1; then > DAYS=$1 > else > DAYS=90 > fi > > SRC_DIR="/var/cache/apt/archives" > DEST_DIR="/backup/debian/packages/old" > TMP_SCRIPT="tmp.sh" > > find "$SRC_DIR" -daystart -mtime +$DAYS -printf "mv \"%p\" \"$DEST_DIR\";\n" > > "$TMP_SCRIPT" > # For deleting the following should work (untested) > #find "$SRC_DIR" -daystart -mtime +$DAYS -printf "rm \"%p\";\n" > > "$TMP_SCRIPT" > > source "$TMP_SCRIPT" > rm "$TMP_SCRIPT" > Very neat. Thanks - Tom George > > -- > Maurits van Rees | http://maurits.vanrees.org/ [Dutch/Nederlands] > Public GnuPG key: keyserver.net ID 0x1735C5C2 > "Let your advance worrying become advance thinking and planning." > - Winston Churchill -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Are /var/cache/apt/archive files essential?
On Fri, 03 Jun 2005 15:44:46 -0400 "Thomas H. George" <[EMAIL PROTECTED]> wrote: > Suddenly I have used up all 9 Gbytes of the /var partition. I > eliminated some of the gzipped log files but that didn't help much. > For the moment I moved all of the /var/cache/apt/archive files - > about 3 Gbytes - to another partition. The system is still working > but I don't know how much damage this might cause in the future. > > I note that there are multiple entries for most of the packages in the > > archive with dates going back two or three years. Perhaps it would > have been better to weed out the older entries. If so, is there a > simple way to do this? > > Tom George > as root: apt-get clean doesn't get much simpler. Cybe R. Wizard -- Q: What's the difference between MicroSoft Windows and a virus? A: Apart from the fact that viruses are supported by their authors, use optimized, small code and usually perform well, none. Winduhs -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
Re: Are /var/cache/apt/archive files essential?
On Fri, Jun 03, 2005 at 03:44:46PM -0400, Thomas H. George wrote: > I note that there are multiple entries for most of the packages in the > archive with dates going back two or three years. Perhaps it would have > been better to weed out the older entries. If so, is there a simple way > to do this? From the apt-get manual page: clean clean clears out the local repository of retrieved package files. It removes everything but the lock file from /var/cache/apt/archives/ and /var/cache/apt/archives/partial/. When APT is used as a dselect(8) method, clean is run automatically. Those who do not use dselect will likely want to run apt-get clean from time to time to free up disk space. autoclean Like clean, autoclean clears out the local repository of retrieved package files. The differ- ence is that it only removes package files that can no longer be downloaded, and are largely useless. This allows a cache to be maintained over a long period without it growing out of control. The configuration option APT::Clean-Installed will prevent installed packages from being erased if it is set to off. So call 'apt-get autoclean' to remove most of your packages or 'apt-get clean' to remove them all. It doesn't hurt. You just need to fetch those files again when you have to reinstall for some reason. Note though that having an older version of a package lying around in the apt cache can be useful when you discover a bug in the new version and want to revert to the old. To have more control, cobble together some shell commands. For example see this posting by me: http://lists.debian.org/debian-user/2004/10/msg01671.html At the end that post boils down to the following shell line that prints all packages that have a more than two versions in that directory. ls /var/cache/apt/archives | sort -g | uniq -c -t "_" -d -W 1 | grep -v -E "^[[:blank:]]*2 " | sed "s/ *[[:digit:]*] \(.*\)$/\1/g" I have another script in /usr/local/bin/cleanpackages that moves old packages to another directory. That should then be a directory on a partition where you have more space. Instead of moving them you can delete them if you want. #! /bin/sh # Move old files to another dir. Mostly meant for the apt cache. if test $# -eq 1; then DAYS=$1 else DAYS=90 fi SRC_DIR="/var/cache/apt/archives" DEST_DIR="/backup/debian/packages/old" TMP_SCRIPT="tmp.sh" find "$SRC_DIR" -daystart -mtime +$DAYS -printf "mv \"%p\" \"$DEST_DIR\";\n" > "$TMP_SCRIPT" # For deleting the following should work (untested) #find "$SRC_DIR" -daystart -mtime +$DAYS -printf "rm \"%p\";\n" > "$TMP_SCRIPT" source "$TMP_SCRIPT" rm "$TMP_SCRIPT" -- Maurits van Rees | http://maurits.vanrees.org/ [Dutch/Nederlands] Public GnuPG key: keyserver.net ID 0x1735C5C2 "Let your advance worrying become advance thinking and planning." - Winston Churchill signature.asc Description: Digital signature
Are /var/cache/apt/archive files essential?
Suddenly I have used up all 9 Gbytes of the /var partition. I eliminated some of the gzipped log files but that didn't help much. For the moment I moved all of the /var/cache/apt/archive files - about 3 Gbytes - to another partition. The system is still working but I don't know how much damage this might cause in the future. I note that there are multiple entries for most of the packages in the archive with dates going back two or three years. Perhaps it would have been better to weed out the older entries. If so, is there a simple way to do this? Tom George -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of "unsubscribe". Trouble? Contact [EMAIL PROTECTED]
