Re: Have suggestions for a "roll your own file server"?
I've been running CentOS NAS servers for the past 15 years. I can usually find a decent rackmount server on eBay for around $99 that I imagine was probably used in the financial industry for a year or so and then turned over to a liquidator for disposal. I then load the drive bays with SATA drives I order from Amazon, install CentOS, and configure the machine as an NFS server. I configure the drives in RAID-1 pairs when installing the OS. On the first pair I create a regular partition for /boot and an LVM pv volume for the rest of the disk, and for the remaining drive pairs I make the entire disk a pv volume. I install a cron job that regularly checks the drives and sends me email alerts if it detects any problems, so I can replace failing drives in a timely manner. I've never calculated how it impacts my electric bill, and it's possible a low-power server box may be more cost effective overall, but my electric bill hasn't been noticeably bad. On Thu, May 27, 2021 at 2:08 PM Tom Buskey wrote: > In my mind, the client is what gets backed up. The host stores it. > I hadn't considered ransomware. I think ZFS snapshots on the host would > work well w/o using much more space. > > @Bruce Labitt I did Solaris up until just after the > Oracle takeover (Solaris 11 even!), including some at home. I've forgotten > just about everything about Solaris that isn't in Linux. If you're not > using it, you lose it. > > On Wed, May 26, 2021 at 4:58 PM Bruce Labitt wrote: > >> Thanks for the compliment. Put a bit of work into it. Self taught hobby >> machinist. Self taught Linux as well. Only have had machines for 18 >> months. Embarrassed to say how long I've been using Linux, as I keep on >> asking basic questions. >> >> On Wed, May 26, 2021, 12:14 PM Tom Buskey wrote: >> >>> My Fedora /etc/fstab has spaces >>> UUID=54103729-6e0a-4345-a2b8-8b8cded29ee1 /boot ext4 >>> defaults1 2 >>> >>> I've had clients initiate rsync for security. I think the client >>> initiation would offload the rsync compute from the server. >>> For a home server, it's nice to just monitor the server instead of >>> multiple clients. >>> >>> Nice buiild >>> >>> On Wed, May 26, 2021 at 11:00 AM Bruce Labitt < >>> bruce.lab...@myfairpoint.net> wrote: >>> Finally back to this. Built a stack of metal plates that house my RPI4, a boot SSD, a 1TB RAID1 array, and both active and passive USB3 hubs. Machined parts so everything is bolted and clamped down. Have a PWM fan that cools the RPI4 proportional to load that runs under systemd. System boots from SSD. (No SD card.) It's kind of a brick sh!thouse, but it's sturdy. Have created the RAID1 device - or it will be finished in 45 minutes. It is still syncing. Now I'd like to add the md0 device to /etc/fstab. The example I see is with the device name. From https://www.tecmint.com/create-raid1-in-linux/ /dev/md0/mnt/raid1ext4defaults0 0 I've read it is better to use the UUID. Is the following the correct syntax? PARTUUID=my_complete_md0_UUID /mnt/raid1ext4defaults 0 0 where my_complete_md0_UUID comes from $ lsblk -o UUID /dev/md0 Does one need to use tabs in fstab, or are spaces ok? Once I figure this out - I have to figure out some rsync magic. Is it better for the server to initiate the rsync, or the remote devices? After all this I have to make another one. That shouldn't take as long as the first time! For some pictures of the hardware build see https://www.hobby-machinist.com/threads/an-rpi4-based-file-server.92273/#post-846939 On 3/10/21 8:49 PM, Bruce Labitt wrote: I'll take a look at that. Thanks for the link. On Wed, Mar 10, 2021 at 8:15 PM Marc Nozell (m...@nozell.com) < noz...@gmail.com> wrote: > Just to put a plug in for a colleague's work: > https://perfectmediaserver.com/It covers everything from disk > purchasing strategies, burn-in, filesystems (ZFS, SnapRAID, etc). > > He also hosts a podcast that folks here may find interesting: > https://selfhosted.show/ > > -marc > > On Wed, Mar 10, 2021 at 8:08 PM wrote: > >> OK: >> >> s/RPi4/some-other-cheap-computer-with-USB-3.x>/g >> >> Unless you build multiple Ethernet or WiFi or LTE modem connections >> your networking will still be the slowest thing. >> >> You do not need huge amounts of CPU power, or huge amounts of RAM. >> >> My basic point is that if you stick with simple RAID (like mirroring) >> but also set up a unit that is remote from your own home you could >> protect >> your own data from fire, flood and theft to a reasonable level and even >> protect your friend's data by backing up their data to your device. >> >> Add snapshots as suggested by Tom Buskey,
Re: Have suggestions for a "roll your own file server"?
In my mind, the client is what gets backed up. The host stores it. I hadn't considered ransomware. I think ZFS snapshots on the host would work well w/o using much more space. @Bruce Labitt I did Solaris up until just after the Oracle takeover (Solaris 11 even!), including some at home. I've forgotten just about everything about Solaris that isn't in Linux. If you're not using it, you lose it. On Wed, May 26, 2021 at 4:58 PM Bruce Labitt wrote: > Thanks for the compliment. Put a bit of work into it. Self taught hobby > machinist. Self taught Linux as well. Only have had machines for 18 > months. Embarrassed to say how long I've been using Linux, as I keep on > asking basic questions. > > On Wed, May 26, 2021, 12:14 PM Tom Buskey wrote: > >> My Fedora /etc/fstab has spaces >> UUID=54103729-6e0a-4345-a2b8-8b8cded29ee1 /boot ext4 >> defaults1 2 >> >> I've had clients initiate rsync for security. I think the client >> initiation would offload the rsync compute from the server. >> For a home server, it's nice to just monitor the server instead of >> multiple clients. >> >> Nice buiild >> >> On Wed, May 26, 2021 at 11:00 AM Bruce Labitt < >> bruce.lab...@myfairpoint.net> wrote: >> >>> Finally back to this. Built a stack of metal plates that house my RPI4, >>> a boot SSD, a 1TB RAID1 array, and both active and passive USB3 hubs. >>> Machined parts so everything is bolted and clamped down. Have a PWM fan >>> that cools the RPI4 proportional to load that runs under systemd. System >>> boots from SSD. (No SD card.) It's kind of a brick sh!thouse, but it's >>> sturdy. Have created the RAID1 device - or it will be finished in 45 >>> minutes. It is still syncing. >>> >>> Now I'd like to add the md0 device to /etc/fstab. The example I see is >>> with the device name. From >>> https://www.tecmint.com/create-raid1-in-linux/ >>> /dev/md0/mnt/raid1ext4defaults0 0 >>> >>> I've read it is better to use the UUID. Is the following the correct >>> syntax? >>> >>> PARTUUID=my_complete_md0_UUID /mnt/raid1ext4defaults 0 0 >>> >>> where my_complete_md0_UUID comes from >>> $ lsblk -o UUID /dev/md0 >>> >>> Does one need to use tabs in fstab, or are spaces ok? >>> >>> Once I figure this out - I have to figure out some rsync magic. Is it >>> better for the server to initiate the rsync, or the remote devices? >>> >>> After all this I have to make another one. That shouldn't take as long >>> as the first time! For some pictures of the hardware build see >>> https://www.hobby-machinist.com/threads/an-rpi4-based-file-server.92273/#post-846939 >>> >>> >>> >>> On 3/10/21 8:49 PM, Bruce Labitt wrote: >>> >>> I'll take a look at that. Thanks for the link. >>> >>> On Wed, Mar 10, 2021 at 8:15 PM Marc Nozell (m...@nozell.com) < >>> noz...@gmail.com> wrote: >>> Just to put a plug in for a colleague's work: https://perfectmediaserver.com/It covers everything from disk purchasing strategies, burn-in, filesystems (ZFS, SnapRAID, etc). He also hosts a podcast that folks here may find interesting: https://selfhosted.show/ -marc On Wed, Mar 10, 2021 at 8:08 PM wrote: > OK: > > s/RPi4/some-other-cheap-computer-with-USB-3.x>/g > > Unless you build multiple Ethernet or WiFi or LTE modem connections > your networking will still be the slowest thing. > > You do not need huge amounts of CPU power, or huge amounts of RAM. > > My basic point is that if you stick with simple RAID (like mirroring) > but also set up a unit that is remote from your own home you could protect > your own data from fire, flood and theft to a reasonable level and even > protect your friend's data by backing up their data to your device. > > Add snapshots as suggested by Tom Buskey,perhaps encryption of file > systems and data-streams and you can have a rather simple, server where > you > learn a lot by planning it out and setting it up rather than buying an > "off > the shelf" solution or simply using a "web backup". > > And good catch on the USB power supply. > > md > > On 03/10/2021 6:53 PM Joshua Judson Rosen > wrote: > > > > > > I'm not sure about the Raspberry Pi 4, but up thru the raspi 3+ > there are... problems, e.g.: > > > > Beware of USB on the raspi: there are some bugs in the silicon that > pretty severely > > cripple performance when multiple `bulk' devices are used at > simultaneously, > > sometimes to the point of making it unusable (e.g. if you want to > use a better Wi-Fi > > adapter/antenna than the one built onto the board, and connect an > LTE modem so that > > your raspi roam onto that if Wi-Fi becomes unavailable, throughput > on whichever of those > > interfaces you're actually using can become abysmal). IIRC the issue > is basically > > that the
