Am Thu, Dec 08, 2022 at 06:36:14PM +0000 schrieb Wols Lists: > > > I've pretty much reached a limit on my backups. I'm up to a 16TB hard > > > drive for one and even that won't last long. Larger drives are much > > > more costly. A must have NAS is quickly approaching. I've been > > > searching around and find some things confusing. I'm hoping someone can > > > clear up that confusion. I'm also debating what path to travel down. > > > I'd also like to keep costs down as well. That said, I don't mind > > > paying a little more for one that would offer a much better option. > > > > > > Path one, buy a NAS, possibly used, that has no drives. If possible, I > > > may even replace the OS that comes on it or upgrade if I can. I'm not > > > looking for fancy, or even RAID. Just looking for a two bay NAS that > > > will work. First, what is a DAS? Is that totally different than a > > > NAS? From what I've found, a DAS is not what I'm looking for since I > > > want a ethernet connection and the ability to control things over the > > > network. It seems DAS lacks that feature but not real sure. I'm not > > > sure I can upgrade the software/OS on a DAS either. > > > […] > > > > DAS is direct-attached-storage. I don't think you want that. > > Depends. If it fits in the safe, and can be connected using one of these > eSATA thingy connectors, it might be a very good choice. > > […] > > I get the impression Dale isn't actually PLANNING his disk storage. It's > just a case of "help I'm downloading all this stuff where do I put it!!!"
Haha, thanks for the laugh. > Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all > this stuff, and plug in and remove drives as required. You don't need all > these huge drives if you think about what you're going to do with it all. That’s actually a good idea. Either use a hot swap frame for an internal 5¼″ PC bay, a desktop dock for bare drives or a multi-bay enclosure. The market is big, you have lots of choices. USB (with or without integrated hub), eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=hddocks Advantages: - no separate system to maintain just for storage: save $$$, time and power - very flexible: no chassis limitation on number of disks - no bulky external enclosures, each using a different power brick and cable - minimum volume to put into a safe (just get or make a bulk storage case) Disadvantages: - not as “fancy” as a NAS - possibly not all disks can be used at the same time - physical handling of naked disks takes more care - LVM is not practical, so use each disk separately - you gotta remember which files are where¹ - SATA connectors aren’t made for very many insertion cycles (I think the spec says 50?), which doesn’t mean they endure much more, but still … > (And while it takes time and hammers the system, I regularly record off the > TV getting a 2GB .ts file, convert it to mp4 - same resolution - and reduce > the size by an order of magnitude - maybe more. Well, ts uses mpeg2 encoding, just like old video DVDs, which is very inefficient when compared with modern h264/h265. Modern digital TV broadcast uses h264 by now. Incidentally, I got myself a new HDD today: an external 2.5″ WD Passport Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also because I need temporary space if I want to convert my NAS RAID-Z2 to Z1. ¹ I do have several external USB disks, plus the big NAS. All of which don’t run very often. And I don’t want to turn them on just to look for a certain file. That’s why I have another little script. ;-) It uses the `tree` command to save the complete content listing of a directory into a text file and names the file automatically by the name of the directory it crawls. So if I want to find a file, I just need to grep through my text files. -- Grüße | Greetings | Salut | Qapla’ Please do not share anything from, with or about me on any social network. The whale is characterised by its bulky form factor.
#!/bin/sh
# history
# 2018-02-10 initial version
# 2020-06-10 added -a option
# ????-??-?? gzip result
# 2021-03-22 put current date into output filename
# 2021-04-06 code refactoring with more functions and input sanitisation
# 2021-12-22 prefer zstd over gzip
# 2022-01-23 don't create symlink if there is no previous file
# 2022-01-30 added xz to compressors
# 2022-12-08 some cleanup, added -C and -K options
die() {
echo "${@}" > /dev/stderr
exit 1
}
usage() {
cat <<-EOF
Usage: $(basename "$0") [-o NAME] [DIR]
A wrapper to tree, it writes the content of DIR into a text file.
The file is named after DIR and the current date, and symlink to the
most recent version is set. The file is automatically compressed to
zstd, xz or gzip, whichever is available in that order.
Options:
-a access attributes: owner, group, permissions
-C do not compress result file
-K do not keep backup of existing file in case of overwriting
-o The destination where to write the trees.
Default: .
If it is a directory: write in there
If it is a filename: use that as base. If not,
use the name of the directory as name base.
EOF
}
test_writable() {
touch "$1" 2>/dev/null || die "Cannot create file in $(dirname "$1")."
}
# run tree and redirect output to destination file
call_to_tree() {
WHICH="$1"
OUTPATH="$2-$1"
shift 2
declare TREE_ARGS
TREE_ARGS+=("$@")
local DATED_PATH="$OUTPATH-$TODAY"
local EXT
local PACK
local CREATE_SYMLINK=no
if [ "$COMPRESS" = "no" ]; then
TREE_ARGS+=("-o")
TREE_ARGS+=("$DATED_PATH")
elif command -v zstd > /dev/null; then
PACK="zstd --rm -q -13"
EXT=".zst"
TREE_ARGS+=("-o")
TREE_ARGS+=("$DATED_PATH")
elif command -v xz > /dev/null; then
PACK="xz"
EXT=".xz"
TREE_ARGS+=("-o")
TREE_ARGS+=("$DATED_PATH")
else
PACK="gzip"
EXT=".gz"
fi
ls "$OUTPATH-*" > /dev/null 2>&1 && CREATE_SYMLINK=yes
# pack yet unpacked file
if [ "$COMPRESS" != "no" ]; then
[ -f "$DATED_PATH" ] && [ ! -f "$DATED_PATH$EXT" ] && $PACK
"$DATED_PATH"
fi
# move away old file
if [ -f "$DATED_PATH$EXT" ]; then
[ "$KEEP_OLD" = "no" ] && rm -f "$DATED_PATH$EXT" || mv -f
"$DATED_PATH$EXT" "$DATED_PATH.old$EXT"
fi
test_writable "$DATED_PATH"
echo "Writing $WHICH tree to $(realpath "$DATED_PATH")$EXT"
BEGIN_TIME=$(date +%s)
"${TREE_ARGS[@]}"
END_TIME=$(date +%s)
echo >> "$DATED_PATH"
df -BM . | awk 'NR==2 { print "Size: " $3 " Available: " $4 " Usage: "
$5; }' >> "$DATED_PATH"
[ "$PACK" ] && $PACK "$DATED_PATH"
if [ "$CREATE_SYMLINK" = yes ]; then
if [ -e "$OUTPATH" ] && [ ! -L "$OUTPATH" ]; then
echo "Cannot set symlink, filename already taken."
else
echo "Setting symlink $OUTPATH -> $(basename "$DATED_PATH")"
ln -sfn "$(basename "$DATED_PATH$EXT")" "$OUTPATH$EXT"
fi
fi
echo "Reading of directory took $((END_TIME - BEGIN_TIME)) seconds."
}
# parse arguments
unset ACCESSRIGHTS
unset COMPRESS
unset KEEP_OLD
while getopts "CKaho:" OPTION
do
case $OPTION in
C) COMPRESS=no ;;
K) KEEP_OLD=no ;;
a) ACCESSRIGHTS=yes ;;
h) usage; exit 0 ;;
o) OUTARG="$OPTARG" ;;
esac
done
shift $((OPTIND-1)); OPTIND=1
# input sanitisation and derivation of destination paths
DIR="${1:-$(pwd)}"
[ -d "$DIR" ] || die "The given directory does not exist."
[ "$OUTARG" ] && OUTARG="$(realpath "$OUTARG")" || OUTARG="$(pwd)"
if [ -d "$OUTARG" ]; then
OUTDIR="$OUTARG"
OUTNAME="$(basename "$(realpath "$DIR")")" # realpath to catch `videotree .`
[ "$OUTNAME" = "/" ] && OUTNAME=root
else
OUTDIR="$(dirname "$OUTARG")"
OUTNAME="$(basename "$OUTARG")"
fi
TODAY="$(date +%F)"
cd "$DIR"
# write a very verbose file with permission and owner information
call_to_tree "detailed" "$OUTDIR/$OUTNAME" tree -af -DFins --dirsfirst
--du --timefmt "%Y-%m-%d %T" ${ACCESSRIGHTS:+-pug}
# write a smaller version with only file size information
#call_to_tree "simple" "$OUTDIR/$OUTNAME-tree" tree -ax -n --du -h --dirsfirst
echo "Done."
signature.asc
Description: PGP signature
