Re: [pfSense] APU and SSD: full install or NanoBSD
-Original Message- From: Jeppe Øland Sent: Wednesday, October 29, 2014 18:46 I've been on an Atom board with a Kingston SSD for like 3 years now ... In that time I've gone through 3 dead SSDs (which Kingston replaced). Due to that I'm now running the nano build ... the SSD seems to hold We use the 32GB sandisk [http://amzn.com/B008U3038I] drives with a nano install, but the slack space is an extra partition which can be used as needed. The logs and pcap data are sent to a external server. -Jason -- -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- - - - Jason Pyeron PD Inc. http://www.pdinc.us - - Principal Consultant 10 West 24th Street #100- - +1 (443) 269-1555 x333Baltimore, Maryland 21218 - - - -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This message is copyright PD Inc, subject to license 20080407P00. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On Oct 30, 2014, at 7:14 AM, Jason Pyeron jpye...@pdinc.us wrote: -Original Message- From: Jeppe Øland Sent: Wednesday, October 29, 2014 18:46 I've been on an Atom board with a Kingston SSD for like 3 years now ... In that time I've gone through 3 dead SSDs (which Kingston replaced). Due to that I'm now running the nano build ... the SSD seems to hold We use the 32GB sandisk [http://amzn.com/B008U3038I] drives with a nano install, but the slack space is an extra partition which can be used as needed. One if the ways that SSD life can be extended is to write less than the full disk. If your device supports it, sometimes these unused sectors can be used for remapping and included in the wear-leveling algorithms. Of course, the nano builds contain an entire partition full of bits that are unlikely to ever be used AND which can't be used as spare blocks (because entirely useless bits occupy your SSD.) Simply using a larger SSD (that has a decent wear-leveling algorithm) will greatly increase the TBW figure. Compression is another tool. Again, fewer bytes written. Finally, the eMMC parts we use on the coming Netgate boards can be put in a mode that halves the capacity in exchange for a 5X increase in write endurance. Just use the nano build isn't going to cut it. Jim ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
3 year old Kingston SSDs are not like new Kingston SSDs. Agreed. On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. As for Nano, I thought it mounted almost everything as RO and only changed settings to write down settings changes, and RRD databases etc on reboots? ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On Oct 30, 2014, at 9:28 AM, Jeppe Øland jol...@gmail.com wrote: 3 year old Kingston SSDs are not like new Kingston SSDs. Agreed. On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. I’m sure I’ve been burned at least as badly by these, and others, and I still buy from them. Samsung 840s are the darling of the “cheap, fast SSD” and they turn out to suck, too: http://www.pcper.com/news/Storage/Samsung-Germany-acknowledges-840-Basic-performance-slow-down-promises-fix As for Nano, I thought it mounted almost everything as RO and only changed settings to write down settings changes, and RRD databases etc on reboots? I think I’ve already responded to this. nano is a 10 year old “solution” to the problems that existed at the time. http://markmail.org/message/rxe4xfpmdwva7q3e http://markmail.org/message/rxe4xfpmdwva7q3e That doesn’t mean it’s a bad solution, but though it’s author is a brilliant individual, he obviously didn’t envision SSD in 2004. Jim ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
Jim Thompson schreef op 30-10-2014 16:33: On Oct 30, 2014, at 9:28 AM, Jeppe Øland jol...@gmail.com mailto:jol...@gmail.com wrote: 3 year old Kingston SSDs are not like new Kingston SSDs. Agreed. On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. I’m sure I’ve been burned at least as badly by these, and others, and I still buy from them. Samsung 840s are the darling of the “cheap, fast SSD” and they turn out to suck, too: http://www.pcper.com/news/Storage/Samsung-Germany-acknowledges-840-Basic-performance-slow-down-promises-fix We have about 70 Dell optiplex desktops that have a Samsung 830 in them that appears to be doing fine. None has failed yet. We also have 300 cash registers running the Intel 320 series 80GB and so far 3 have failed in 8MB mode, eventhough they do have the correct firmware. It's basically the way it tells you something went wrong. We are very picky about our Intel SSD models, only a few have power protection circuits. Basically only the models with the in-house Intel controller have this. (X25-M, Intel 320, Intel S3500/S3700). We did have 1 OCZ Vertex 2 that predictably died just after the 1st year in a developers laptop, that was a train wreck waiting to happen, and it did. Another production box is a 12 disk Raid 6 (~2TB) with 300GB Intel 320 series, it's been fine on a light write workload. (70/30). Cheers, Seth ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On Thu, Oct 30, 2014 at 8:33 AM, Jim Thompson j...@smallworks.com wrote: On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. I’m sure I’ve been burned at least as badly by these, and others, and I still buy from them. What can you do? The speed increase from SSDs in a PC means its almost impossible to go back to an HDD. And in a firewall/appliance, the benefits from no moving parts/lower power/heat/noise is hard to ignore. As for Nano, I thought it mounted almost everything as RO and only changed settings to write down settings changes, and RRD databases etc on reboots? I think I’ve already responded to this. nano is a 10 year old “solution” to the problems that existed at the time. http://markmail.org/message/rxe4xfpmdwva7q3e That doesn’t mean it’s a bad solution, but though it’s author is a brilliant individual, he obviously didn’t envision SSD in 2004. Are you saying the nano release only covers the boot-slices? I thought the nano/embedded versions also write less to the disk. I don't have a full install handy to check, but the nano install definitely mounts the drive RO, and all runtime stuff (/var, /tmp) is run out of RAM disks. Regards, -Jeppe ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
[pfSense] Install CD - I don't know where to go with this
I'm trying to make an install CD but no joy. Upfront, this is not a pfSense issue but maybe someone can help. Thanks to those who have already responded. I used WinISO, which lets me fiddle with the boot record, so I burned a CD and then made an ISO from it and the ISO has a boot record. But it won't boot. I went through the same exercise with Oracle Linux and got the same results. The same machine boots up a Windows OS just fine. I'm trying to boot onto a DL380 G3 ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
Every data I've seen on them sucking has to do specifically with NTFS, which the newly released firmware update is supposed to fix. We are using 840Evo's in all of our storage arrays, and haven't seen any issues(EXT4/ZFS). Josh Reynolds, Chief Information Officer SPITwSPOTS, www.spitwspots.com http://www.spitwspots.com On 10/30/2014 07:33 AM, Jim Thompson wrote: On Oct 30, 2014, at 9:28 AM, Jeppe Øland jol...@gmail.com mailto:jol...@gmail.com wrote: 3 year old Kingston SSDs are not like new Kingston SSDs. Agreed. On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. I’m sure I’ve been burned at least as badly by these, and others, and I still buy from them. Samsung 840s are the darling of the “cheap, fast SSD” and they turn out to suck, too: http://www.pcper.com/news/Storage/Samsung-Germany-acknowledges-840-Basic-performance-slow-down-promises-fix As for Nano, I thought it mounted almost everything as RO and only changed settings to write down settings changes, and RRD databases etc on reboots? I think I’ve already responded to this. nano is a 10 year old “solution” to the problems that existed at the time. http://markmail.org/message/rxe4xfpmdwva7q3e That doesn’t mean it’s a bad solution, but though it’s author is a brilliant individual, he obviously didn’t envision SSD in 2004. Jim ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On Oct 30, 2014, at 3:39 PM, Dave Warren da...@hireahit.com wrote: On 2014-10-30 13:06, Jeppe Øland wrote: On Thu, Oct 30, 2014 at 8:33 AM, Jim Thompsonj...@smallworks.com wrote: On the other hand, I tend to distrust manufacturers that shipped completely unreliable drives without any thought. Kingston/OCZ/Crucial are all in this boat for me. I’m sure I’ve been burned at least as badly by these, and others, and I still buy from them. What can you do? Buy quality instead of junk? I've been burned by OCZ and Crucial for sure (including silent write failures!), although I'm not sure I've ever had a Kingston. http://techreport.com/review/26058/the-ssd-endurance-experiment-data-retention-after-600tb tl;dr: Buy Intel, or very specific Samsung SSDs. For non-endurance testing, you'll have better reliability out of a modern, quality SSD than rotational drives, both on a per-drive and per-GB basis. We’ve already shown that specific Samsung SSDs are flawed. Others have already pointed out that not all “Intel” SSDs are created equal. We’re using Kingston eMMCs on the coming netgate hardware. We ship *specific* Intel SSDs (purchased in volume) for those systems sold with an SSD in the pfSense Store. The speed increase from SSDs in a PC means its almost impossible to go back to an HDD. And in a firewall/appliance, the benefits from no moving parts/lower power/heat/noise is hard to ignore. There are use cases for rotational drives, primarily where $/GB is a factor and performance isn't, but I tend toward small SSDs over rotational drives unless there is a strong use-case for bulk storage. I really can't imagine using a workstation without a SSD as primary storage though, I just don't have the patience. Even a cheapo 30GB/60GB/whatever SSD is more than enough for pfSense and makes a far more reliable solution than external flash. I strongly disagree.SSDs have to be part of a system, especially in an embedded environment. The debacle with the “cheap 30GB” m-sata drive from PC Engines earlier in the year (they had to take them all back) should amply demonstrate why thinking such as what you express here is deeply flawed. I’m getting a bit tired of the “shove a bunch of components together; expect it to work; complain about pfSense when it doesn’t” approach shown by some in the community. You can do what you wish, of course. You don’t *have* to be solutions from pfSense, but pfSense solutions are “best of breed” (given certain constraints). We definitely don’t buy “cheapo xx/yy/whatever SSDs”. The big reason for this is that the consequences for “you” (the royal you) being wrong are a few hundred dollars. The consequences for us being wrong can run to hundreds of thousands of dollars (or higher). “Oh crap, my SSD failed” takes on a whole new meaning when you realize that there are thousands more in the world that are about to suffer the same fate, and you offered a warranty. The “use case” for rotational drives is still present for high-write environments. (I was just discussing this with a customer at lunch today.) Jim ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On 2014-10-30 17:15, Jim Thompson wrote: On Oct 30, 2014, at 3:39 PM, Dave Warren da...@hireahit.com wrote: Buy quality instead of junk? ... Even a cheapo 30GB/60GB/whatever SSD is more than enough for pfSense and makes a far more reliable solution than external flash. I strongly disagree.SSDs have to be part of a system, especially in an embedded environment. The debacle with the “cheap 30GB” m-sata drive from PC Engines earlier in the year (they had to take them all back) should amply demonstrate why thinking such as what you express here is deeply flawed. Sorry if I wasn't clear, I meant a cheapo SSD because it's small -- I'm suggesting you don't need to invest in a large or fast SSD for pfSense, but rather, cheap out on size, while getting a quality device built for lifespan and reliability. -- Dave Warren http://www.hireahit.com/ http://ca.linkedin.com/in/davejwarren ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] Install CD - I don't know where to go with this
Sounds like the CD drive in that server probably isn't working right (assuming it is configured in the BIOS to boot from it), try an external CD-ROM, or use the memstick image to boot from USB flash instead. On Thu, Oct 30, 2014 at 6:01 PM, Mark Hisel mark_hi...@yahoo.com wrote: I'm trying to make an install CD but no joy. Upfront, this is not a pfSense issue but maybe someone can help. Thanks to those who have already responded. I used WinISO, which lets me fiddle with the boot record, so I burned a CD and then made an ISO from it and the ISO has a boot record. But it won't boot. I went through the same exercise with Oracle Linux and got the same results. The same machine boots up a Windows OS just fine. I'm trying to boot onto a DL380 G3 ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
Things will get outrageous soon with the advent of M.2 PCI SSDs on a x4 connection. The speeds of m.2 on x4 do look amazing, but the prices and sizes of them probably means that not many people will be tossing them into their firewalls anytime soon. For projects like firewalls, and to act as server boot drives, I use 60GB ssds that I find on sale. With 60, 120, etc. sata drives you get the latest technologies. I've owned and installed almost every brand over the last few years, and have only had one OCZ drive fail. The first two ssd's I purchased were 60GB Vertex 2 drives that still work fine. Of course, you deal with far more of them than I do, but I trust SSDs as much as hard drives. By the way, I use zfs on several large arrays, and don't see why anyone is against it. Guess I missed the discussion. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
Re: [pfSense] APU and SSD: full install or NanoBSD
On Oct 30, 2014, at 7:35 PM, Dave Warren da...@hireahit.com wrote: On 2014-10-30 17:15, Jim Thompson wrote: On Oct 30, 2014, at 3:39 PM, Dave Warren da...@hireahit.com wrote: Buy quality instead of junk? ... Even a cheapo 30GB/60GB/whatever SSD is more than enough for pfSense and makes a far more reliable solution than external flash. I strongly disagree.SSDs have to be part of a system, especially in an embedded environment. The debacle with the “cheap 30GB” m-sata drive from PC Engines earlier in the year (they had to take them all back) should amply demonstrate why thinking such as what you express here is deeply flawed. Sorry if I wasn't clear, I meant a cheapo SSD because it's small -- I'm suggesting you don't need to invest in a large or fast SSD for pfSense, but rather, cheap out on size, while getting a quality device built for lifespan and reliability. Understood, but even here your suggestion is out of date with respect to the current state-of-the-art.Assuming a decent wear-leveling implementation, larger drives will last longer for a given amount of data written. In the same way that, when flying an airplane you can trade altitude for glide, with modern SSDs, you can trade capacity for endurance. (It also matters *how* you write the data.) In the below, I’m quoting JEDEC-219 compliant numbers/stats. Here’s an equation you might want to think about. Total writes to the device = (Max endurance cycles) * (total partition capacity) / (WAF) Where Maximum Endurance Cycles = the total number of program erase cycles each block in the NAND flash can withstand. For the current generation of MLC flash this is 3,000 Program-Erase Cycles. Write Amplification Factor (WAF) = is a result of wear leveling activity to some degree and the nature of writes to the flash. The actual nand flash is written in units of pages. For the current generation of flash, this page size is typically 16K Bytes. If the nature of writes are sequential within the 16K page, then the WAF should be low. However if this write information is not contiguous, or is interrupted by another write stream then the partial page will be programmed to the NAND flash. In general, random writes will contribute to higher WAF. Ideally we would want WAF to be 1. However, this is the real world, and we have seen this go as high as 20 in some applications with non-ideal writing behavior. (Very poorly behaved, always non-contiguous or interrupted write streams, e.g. logging or sql databases.) Example: Application that writes 100 MB of data to the device per day. 100 MB / day * 365 days / year = 36.5 GB / year Let’s assume a standard mode 4GB CF card/USB/… with perfect wear-leveling (LOL!): Best case: For WAF = 1, standard mode 30GB part: Total Writes = (3,000) * (4GB) / 1 = 12 TB With the above data this yields: 12,000 / 36.5 = 329 years Worst case: WAF = 20, standard mode 4GB part: Total Writes to reach endurance = (3,000) * (4GB) / 20 = 600 GB of data written will exceed endurance With the above data this yields: 600 / 36.5 = 16.4 years This is how a “commodity” flash/SSD vendor (or a shill^W “technology journalist”) will talk to you: “It will take more than 16.4 years to wear out the disk!” The reality is that with the 3000 program-erase cycles rating of today's underlying MLC cells, the 30GB part can support a worst case 600GB of data writes assuming very poorly behaved, always non-contiguous or interrupted write streams. Best case assuming purely contiguous writes would be 12TB. Actual worst case without effective wear-leveling (as was the case with CF cards and a lot of the early SSDs) would be 3,000 writes to a single 16K page. (Thus the “don’t swap to an SSD!” advice so often heard.) Do this, and “Boom!” the sector is dead (or will be quite soon.) If this was in a file that you needed (or worse, a filesystem metadata block), *poof* goes your data. Bummer, dude. This is *also* why SLC flash is often recommended for applications that require high write-endurance. SLC flash can endure approximately 10X the program-erase cycles of MLC flash in a given lithography. The direct result is that today you see a lot of people attempting to quote “TBW” (terabytes written) when talking about SSD / flash endurance, but even then they don’t talk about WAF very often. Once you start thinking about it, it’s not very difficult to figure out that it doesn’t take long to write 600GB on a very busy system, that does a lot of short writes due to logging, etc. Now go run the numbers yourself for a larger SSD (and you can assume wear-leveling). Double the size of the device, and you’ll double the TBW figure, assuming everything else stays the same. Larger density devices will yield correspondingly higher total write endurance since (QED!) they have more blocks of NAND in them. Here is the kicker: the eMMCs we’re using on the coming Netgate hardware (that yes,
Re: [pfSense] APU and SSD: full install or NanoBSD
On Oct 30, 2014, at 8:00 PM, compdoc comp...@hotrodpc.com wrote: Things will get outrageous soon with the advent of M.2 PCI SSDs on a x4 connection. The speeds of m.2 on x4 do look amazing. Now explain why a M.2 PCIe x4 SSD would be more expensive than a M.2 SATA SSD. ___ List mailing list List@lists.pfsense.org https://lists.pfsense.org/mailman/listinfo/list
