Re: [Ldsoss] NAS and Firewall Hardware / Software
Yes, actually running the proxy on your Linksys router would be most likely too much for it to handle. Not to mention, there is really not an easy way to add a hard drive to these if you plan on doing web caching as well. FYI, here is a little howto on how to transparently redirect web traffic to your proxy from a Linksys WRT54G (or something similar) using DD-WRT: http://www.dd-wrt.com/wiki/index.php/Squid_Transparent_Proxy -stacey. Kevin Wise wrote: Thanks for all the input. I actually already have a Linksys router for a hardware firewall, and I wasn't planning to replace that. For the two of you that recommended splitting the hardware, would it change your mind if I limited the new machine to content filtering proxy and NAS (and leave the firewall and NAT to the router)? Kevin ___ Ldsoss mailing list Ldsoss@lists.ldsoss.org http://lists.ldsoss.org/mailman/listinfo/ldsoss
Re: [Ldsoss] NAS and Firewall Hardware / Software
Thanks for all the input. I actually already have a Linksys router for a hardware firewall, and I wasn't planning to replace that. For the two of you that recommended splitting the hardware, would it change your mind if I limited the new machine to content filtering proxy and NAS (and leave the firewall and NAT to the router)? Kevin Scott Barber wrote: I'd second the recommendation to split them up. I run IPCop for the firewall, NAT, etc. and FreeNAS for the network storage. -Scott On 5/15/07, Shawn Willden [EMAIL PROTECTED] wrote: On Tuesday 15 May 2007 12:26:08 am Kevin Wise wrote: I'd really like one piece of hardware that does both. In my mind this would save me maintenance (fewer patches to apply) and maybe even cost. Any comments? I can see value in separating firewall and other functionality, but I personally use one system for both, for just this reason. I Is 512 MB of RAM enough? Plenty. I have 1 GiB in mine, but that's mostly because I had extra RAM lying around from upgrading another box. Should I get hardware RAID or software RAID? In terms of importance to me, reliability is second only to cost. I don't want my files to disappear because my single RAID controller failed and the drive is unreadable by another controller. I use software RAID primarily for this reason, but there are other reasons as well. A big one is flexibility. With Linux MD RAID you can mix and match drives of different types and sizes with no problem, and you can use as many disks as you can pack into the box. I also made use of MD RAID's flexibility to make adding new disks easier. I partitioned my drives into small (50GB) pieces and constructed multiple arrays (each array element on a different disk, obviously), then combined the RAID arrays into a large storage pool with LVM. That way, when I need to add another disk I can add it to the running system by: 1. Pick one physical volume (which is a RAID array) and use pvmove to migrate all of the data off of it. 2. Remove the array from the volume group 3. Destroy the array and rebuild it, adding another partition from the new disk. 4. Add the resulting (larger) physical volume back into the volume group. 5. Go back to step 1, until all arrays have been upgraded. This approach takes a long time, but it's perfectly safe -- after a power outage pvmove picks up right where it left off, yes I have firsthand experience -- and the system continues running and serving files the whole time. Last time I did it, I wrote a script to perform the operations. The script took about 30 minutes to write and about four days to run. Supposedly, someone is looking into giving MD the native ability to add another drive into RAID-5 arrays, which would make the partitioning + LVM stuff less necessary, but it hasn't happened yet. One other thing to consider with your RAID configuration is hot spare vs. RAID 6. I use a hot spare, but I'm planning on rebuilding my system with RAID 6 (one partition array at a time). The odds of two drives failing at once are negligibly small, but I had a scare a few weeks ago when one of the RAID 5 drives failed and while the system was rebuilding onto the hot spare, another drive had some transient error -- I think caused by a SATA controller driver bug, but I can't be sure. The problem with RAID 5 is that the process of rebuilding a degraded RAID 5 array is very intense, so if you have another drive with any latent problems, they'll probably crop up then -- the worst possible time. I think I did the best possible thing I could do -- I immediately shut the machine down (and told the kids the video server was down, possibly for good) and thought things over for a full week. I realized that if I could forcibly reconstruct each array with the exact sequence of drives that were running when the second failure occurred, I might be able to get it back. Luckily, mdadm had e-mailed me the contents of /proc/mdstat, and that had the information I needed. So I powered the machine back up, forcibly rebuilt an array (still in degraded mode) with --assume-clean, then added the spare and crossed my fingers while it recalculated parity and changed to non-degraded mode. When that worked, I repeated with each of the other arrays, then held my breath while I reactivated LVM and then ran fsck on the file systems. It worked and I didn't lose anything. After that harrowing experience, I made two decisions: 1) I need to be more diligent about backing up my important data. I had most of it, but not all of it. 2) I'm going to move to RAID 6 so that I can take two *simultaneous* disk failures and not lose anything. That's better than RAID 5 with a hot spare, and much better than RAID 5 without a spare. BTW, my system has 4 PATA and 2 SATA drives: 3 200 GB PATA 1 250 GB PATA 2 500 GB SATA I have four PATA controllers (two on the mobo, two on a PCI card), so each drive is a
[Ldsoss] NAS and Firewall Hardware / Software
I was wondering if anyone out there has some suggestions for me. You can reply to the list or to me personally ([EMAIL PROTECTED]) if you think your responses would not be appropriate for the whole list. And let me apologize in advance for the length of this post... I'm trying to solve several problems at once (seems to me like many other people face these same problems). Here are my needs (for use in my home network): -- a fault-tolerant (RAID 5 or mirrored) network drive, say 250 GB - 1 TB, available from Windows -- web content filtering proxy / firewall Now I know we've talked about specific software solutions (such as Samba or Dan's Guardian) on this list before. If you have suggestions on what software I might use, I'm all ears. But I'm mostly interested in what kind of hardware I might get or reuse to make this work. I'm pretty sold on Linux for the OS, but there are still a blinding array of possibilities, even if I ignore the choice of distribution. For example, my co-worker has installed Linux on his Linksys router for his firewall, and Linux on a NAS server for his network attached storage. I'm not quite as adventurous, and I'd really like one piece of hardware that does both. In my mind this would save me maintenance (fewer patches to apply) and maybe even cost. Any comments? I was thinking about getting a new bare-bones Intel or AMD system. One problem I have with that is that every computer I've purchased so far is from Dell or Apple, and neither offers a system without the OS. Any suggestions on where I might turn to get a Linux-ready system (need not be pre-installed) suitable for the above purposes, preferably for less than $300? I'm also looking for input on what the system requirements would be for such a system. Is 512 MB of RAM enough? Should I get hardware RAID or software RAID? In terms of importance to me, reliability is second only to cost. I don't want my files to disappear because my single RAID controller failed and the drive is unreadable by another controller. Another choice is to reuse an old computer (Pentium 4, 2.2 GHz). It does not have a SATA controller, so I'd be stuck buying new ATA drives (it currently has 2 80GB drives, which is really not enough for what I want to do). Would I be crazy to use something that old from a reliability standpoint? Also seems like a waste to buy new ATA drives (are they even available any more?). Another option of course is to buy a SATA controller card. Any idea how much that might cost? I look forward to your replies. I'm open to any ideas you might have on how to solve these problems. Kevin Wise ___ Ldsoss mailing list Ldsoss@lists.ldsoss.org http://lists.ldsoss.org/mailman/listinfo/ldsoss
RE: [Ldsoss] NAS and Firewall Hardware / Software
I recently used OpenWRT (http://en.wikipedia.org/wiki/OpenWrt) using the White Russian (named for alcohol) release. It worked okay, but seemed to have some issues. Namely, the WiFi subsystem would not stay disabled. I could disable it via the GPIO, but it would somehow always come back on. The other issue was that there is no un-installation procedure. There are a few listed (jumping hardware pins and rebooting), but they didn't work. The installation of the WR release is done via a hacked bug in the ping routine. This made returning to Linksys firmware impossible for my hardware version (later HW, bug was slightly different). So, be very careful about selecting the right version of hardware before you start. Then again, since you have the source code, you can always change it... ;-) On a side note, it appears that MS has been taking lessons from the RIAA (http://money.cnn.com/magazines/fortune/fortune_archive/2007/05/28/100033867 /). The last 4-5 paragraphs elude to some very interesting plays to come Steve -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kevin Wise Sent: Tuesday, May 15, 2007 12:26 AM To: LDS Open Source Software Subject: [Ldsoss] NAS and Firewall Hardware / Software I was wondering if anyone out there has some suggestions for me. You can reply to the list or to me personally ([EMAIL PROTECTED]) if you think your responses would not be appropriate for the whole list. And let me apologize in advance for the length of this post... I'm trying to solve several problems at once (seems to me like many other people face these same problems). Here are my needs (for use in my home network): -- a fault-tolerant (RAID 5 or mirrored) network drive, say 250 GB - 1 TB, available from Windows -- web content filtering proxy / firewall Now I know we've talked about specific software solutions (such as Samba or Dan's Guardian) on this list before. If you have suggestions on what software I might use, I'm all ears. But I'm mostly interested in what kind of hardware I might get or reuse to make this work. I'm pretty sold on Linux for the OS, but there are still a blinding array of possibilities, even if I ignore the choice of distribution. For example, my co-worker has installed Linux on his Linksys router for his firewall, and Linux on a NAS server for his network attached storage. I'm not quite as adventurous, and I'd really like one piece of hardware that does both. In my mind this would save me maintenance (fewer patches to apply) and maybe even cost. Any comments? I was thinking about getting a new bare-bones Intel or AMD system. One problem I have with that is that every computer I've purchased so far is from Dell or Apple, and neither offers a system without the OS. Any suggestions on where I might turn to get a Linux-ready system (need not be pre-installed) suitable for the above purposes, preferably for less than $300? I'm also looking for input on what the system requirements would be for such a system. Is 512 MB of RAM enough? Should I get hardware RAID or software RAID? In terms of importance to me, reliability is second only to cost. I don't want my files to disappear because my single RAID controller failed and the drive is unreadable by another controller. Another choice is to reuse an old computer (Pentium 4, 2.2 GHz). It does not have a SATA controller, so I'd be stuck buying new ATA drives (it currently has 2 80GB drives, which is really not enough for what I want to do). Would I be crazy to use something that old from a reliability standpoint? Also seems like a waste to buy new ATA drives (are they even available any more?). Another option of course is to buy a SATA controller card. Any idea how much that might cost? I look forward to your replies. I'm open to any ideas you might have on how to solve these problems. Kevin Wise ___ Ldsoss mailing list Ldsoss@lists.ldsoss.org http://lists.ldsoss.org/mailman/listinfo/ldsoss ___ Ldsoss mailing list Ldsoss@lists.ldsoss.org http://lists.ldsoss.org/mailman/listinfo/ldsoss
Re: [Ldsoss] NAS and Firewall Hardware / Software
On Tuesday 15 May 2007 12:26:08 am Kevin Wise wrote: I'd really like one piece of hardware that does both. In my mind this would save me maintenance (fewer patches to apply) and maybe even cost. Any comments? I can see value in separating firewall and other functionality, but I personally use one system for both, for just this reason. I Is 512 MB of RAM enough? Plenty. I have 1 GiB in mine, but that's mostly because I had extra RAM lying around from upgrading another box. Should I get hardware RAID or software RAID? In terms of importance to me, reliability is second only to cost. I don't want my files to disappear because my single RAID controller failed and the drive is unreadable by another controller. I use software RAID primarily for this reason, but there are other reasons as well. A big one is flexibility. With Linux MD RAID you can mix and match drives of different types and sizes with no problem, and you can use as many disks as you can pack into the box. I also made use of MD RAID's flexibility to make adding new disks easier. I partitioned my drives into small (50GB) pieces and constructed multiple arrays (each array element on a different disk, obviously), then combined the RAID arrays into a large storage pool with LVM. That way, when I need to add another disk I can add it to the running system by: 1. Pick one physical volume (which is a RAID array) and use pvmove to migrate all of the data off of it. 2. Remove the array from the volume group 3. Destroy the array and rebuild it, adding another partition from the new disk. 4. Add the resulting (larger) physical volume back into the volume group. 5. Go back to step 1, until all arrays have been upgraded. This approach takes a long time, but it's perfectly safe -- after a power outage pvmove picks up right where it left off, yes I have firsthand experience -- and the system continues running and serving files the whole time. Last time I did it, I wrote a script to perform the operations. The script took about 30 minutes to write and about four days to run. Supposedly, someone is looking into giving MD the native ability to add another drive into RAID-5 arrays, which would make the partitioning + LVM stuff less necessary, but it hasn't happened yet. One other thing to consider with your RAID configuration is hot spare vs. RAID 6. I use a hot spare, but I'm planning on rebuilding my system with RAID 6 (one partition array at a time). The odds of two drives failing at once are negligibly small, but I had a scare a few weeks ago when one of the RAID 5 drives failed and while the system was rebuilding onto the hot spare, another drive had some transient error -- I think caused by a SATA controller driver bug, but I can't be sure. The problem with RAID 5 is that the process of rebuilding a degraded RAID 5 array is very intense, so if you have another drive with any latent problems, they'll probably crop up then -- the worst possible time. I think I did the best possible thing I could do -- I immediately shut the machine down (and told the kids the video server was down, possibly for good) and thought things over for a full week. I realized that if I could forcibly reconstruct each array with the exact sequence of drives that were running when the second failure occurred, I might be able to get it back. Luckily, mdadm had e-mailed me the contents of /proc/mdstat, and that had the information I needed. So I powered the machine back up, forcibly rebuilt an array (still in degraded mode) with --assume-clean, then added the spare and crossed my fingers while it recalculated parity and changed to non-degraded mode. When that worked, I repeated with each of the other arrays, then held my breath while I reactivated LVM and then ran fsck on the file systems. It worked and I didn't lose anything. After that harrowing experience, I made two decisions: 1) I need to be more diligent about backing up my important data. I had most of it, but not all of it. 2) I'm going to move to RAID 6 so that I can take two *simultaneous* disk failures and not lose anything. That's better than RAID 5 with a hot spare, and much better than RAID 5 without a spare. BTW, my system has 4 PATA and 2 SATA drives: 3 200 GB PATA 1 250 GB PATA 2 500 GB SATA I have four PATA controllers (two on the mobo, two on a PCI card), so each drive is a master, for better performance. I use 200 GB of five of the six drives for the main RAID 5 arrays, so I have 800 GB of usable storage there. One of the 200s is the hot spare. The 500 GB drives have 300 left over, so I mirrored that, for another 300 GB usable. All of that storage is in one big 1.1 TB volume. The 50 GB left over on the 250 GB drive is in a separate volume group, with bits carved out for various temp storage uses. So I'm wasting 200 (hot spare) + 200 (parity on RAID-5) + 300 (mirrored) = 700. I'm soon going to add another 500.
