Hi Reese,

Good to hear from you.  Replies inline.

On 5/28/2014 6:06 PM, Reese Johnson wrote:

1. data security ssd features you need, but aren't cheap (Ron
Frazier (TECHC))


----------------------------------------------------------------------

  Message: 1 Date: Tue, 27 May 2014 19:38:03 -0400 From: "Ron
Frazier (TECHC)"<[email protected]>  To: Tech Chat
List<[email protected]>  Subject: [tech-chat] data security
ssd features you need, but aren't cheap Message-ID:
<[email protected]>
Content-Type: text/plain; charset=UTF-8

Hi all,

(Also posting on DC404)

I've been watching the SSD market evolve for a while, and wishing I
could buy one of decent size.  I periodically study the market,
product, specs, and prices to see how things are going.

Being an engineer who's paranoid about the security, longevity, and
reliability of my data storage; I believe that the devil is in the
details, and some of the details of SSD failure modes have always
bugged me.

In my latest review of the state of affairs, I've noted that SSD
prices have dropped to very attractive ranges in the sub $ 0.50 /
GB range.  But, I've also discovered that the really cheap drives
are missing some key features that I'd personally want to see in my
drives.  These are things you have to pay extra for.  For the mix
of features I'd like to see, I've come to realize that I'll have to
pay $ 1 - $ 2 / GB (in current market conditions).  This either
means I wait and save up longer, or buy smaller, or both.  I wanted
to share what I've found.

While at least two of these features are security specific, in my
mind, they're almost all related to data security, in that, if your
drive doesn't reliably store your data for the time period that you
need it to, then your data is not very secure.

Caveat 1, almost everything here is my opinion.  I will talk about
some facts, design options, specs, etc.  But, what you want to do
about those facts, what you think about whether one particular
design element is there or not, is up to you.  Only you can say
what kind of risk factors or implications you think that has for
you.

Caveat 2, this is not meant to be a rigorously documented white
paper.  It's meant to be a synopsis of my observations after
reading a fair number of ads, articles, and data sheets.  As far as
I know, everything I'm saying is true.

Here are 23 important or critical features or specs that I would
want to see in my SSD, if not all of them, at least most of them.
The main point is that you just won't get most of these on a cheap
drive.  And, you won't necessarily get them on an expensive drive.
You have to read the spec sheets and reviews to find out.  Many of
these things are described in Wikipedia, should you be inclined to
look.

Here's what's discussed below, is substantial detail:

01) Data Endurance
Some folks say don't use more than 75% of a ssd for optimal performance

I've heard similar things with varying numbers. I haven't been able to confirm or deny them. Most drives are over provisioned by about 7%, but I don't know what effect that filling up a drive to its reported capacity has.

02) Data Retention 03) SLC or MLC, not TLC flash cells 04) 5 Year
Warranty 05) Overprovisioning 06) TRIM Command
Some say using TRIM with crypto weakenss it.

I was actually discussing the security related aspects of my post with someone on DC404. This came up. Apparently, if you're using an encrypted ssd, and trim is active, all sectors which have been trimmed show up as zeros and are visible to an attacker as such, if I understood correctly. An attacker would be able to see how many sectors you're not using and how that changes over time if they get another look at the drive later. Doesn't sound like too big a deal to me, but I guess it is a form of information leakage. The encrypted parts should still be secure.

07) Background Garbage Collection 08) Power Loss Protection 09)
SMART - Self-Monitoring, Analysis and Reporting Technology 10) Wear
Indicator, Life Remaining, Data Written 11) Exotic Diagnostic Data
12) Wear Leveling 13) NCQ - Native Command Queuing 14) Full Drive
Encryption
I have a Intel 180 GB running FreeBSD 10 setup with UFS and geli. The
cpu is a intel atom not the fastest but I have noticed a bottleneck.
aesni helps with the speed. Have not done any benchmarks but I run
AES-XTS 256 for system and AES-XTS 128 for swap. I've switched to Arch
on my home computers and have setup encrypted LLVM with dm-crypt. I
like FreeBSD's system better if I had to pick. The trick here is
having a unencrypted /boot and using a symlink. You might want to look
at BTRFS also you can use a single btrfs partiton with subvolumes.
Speaking about crypto alot of chatter about truecrypt today.
http://truecrypt.sourceforge.net/? I don't have anything sensitive to
protect. Just worried about a junkie borrowing mine without asking.

I've been looking into the potential issues with using either hardware (in the drive) encryption or software encryption ah la TrueCrypt. I'm not saying which side of the hardware vs software encryption fence I'd fall on, but I found a forum discussion on exactly that:

http://www.wilderssecurity.com/threads/truecrypt-versus-built-in-ssd-encryption.344947/

Some of it is just noise but there are some cool tidbits.

Post 25 by dantz talks about how easy it is for a user to blow up his TrueCrypt data.

And, on page 2:

http://www.wilderssecurity.com/threads/truecrypt-versus-built-in-ssd-encryption.344947/page-2

Post 27 by T-RHex talks about some TrueCrypt best practices and precautions.
Post 30 by dantz talks some more about how to blow up your TrueCrypt data.

Post 29 by LockBox is interesting. He's apparently a strong proponent of hardware encryption.

He mentions these encrypting drives:

http://www.integralmemory.com/product/crypto-ssd-hardware-encrypted-sata-ii-25 http://www.amazon.com/s/ref=nb_sb_noss_2?url=search-alias%3Daps&field-keywords=integral%20crypto%20ssd

The drive has mandatory encryption with Windows (assuming you activate it), can have an admin and user password if desired, and blows your data away when you enter from 6 to 20 wrong passwords. That last thing could be a problem or a feature, depending on your point of view.

I think I'd be inclined to set both passwords and put one on a physical piece of paper in a physical safe. Of course, gotta remember THAT combination.

Obviously, there are many factors to consider. I'm back to generally running Windows these days but do dual boot with Mint (based on Ubuntu) periodically, so that's another factor to consider.

I was just recently reading the TrueCrypt website a day ago. I can't believe that the current version of the website is official. Searching google for truecrypt and limiting to the last 24 hours gives lots of speculation to read.

15) Extra Robust ECC (Error Correction Code) 16) Compressible and
Uncompressible Performance 17) Secure Erase
/dev/random or /dev/urandom with dd?

This came up in the discussion on DC404 too. The other party gave me a link to this white paper:

http://static.usenix.org/event/fast11/tech/full_papers/Wei.pdf

The unfortunate bottom line is that it's IMPOSSIBLE to reliably and completely sanitize an ssd. The overwriting all the sectors method which you mention doesn't work reliably because, due to wear leveling, there can be as many as 16 copies of any given file floating around based on research. Because of overprovisioning, there are many sectors you never have access to from the sata interface, particularly if they're reallocated. Therefore, overwriting doesn't guarantee that the data is gone. Having said that, it does put a big dent in it. I'm clearing free space on an old memory card I'm about to throw away. Doing that a few times should substantially reduce the likelihood that anyone could get data from it. Come to think of it, I may break it in half with pliers before throwing it away. Some drives have secure erase built in, but that doesn't always work due to design flaws. The only way I'm aware of to make sure you can safely give, sell, or lose an ssd at a later point in time is to encrypt it right from the start, and never let it see unencrypted data, and never let your password get loose.

18) Data Center Design 19) SATA 3 - 6 Gbps Interface 20) Data
Cloning Software (4K Sector Support) 21) Physical Size - 2.5" x
7mm 22) 3.5" Mounting Adapter
If you buy a desktop ssd "kit" they have adapters.
23) Intel or Samsung Preferably
I own a Intel and Samsung I'd pot more money on Intel pushing upstream
drivers. I just looked at newegg and feel sick. I paid the price for
my 180GB they are asking for a 1TB SSD now. Tosshiba bought OCZ after
they went under, maybe they will come out with some cool new stuff.
Don't know much about the other points.  One of the resons I swtiched
to Arch is the wiki. https://wiki.archlinux.org/index.php/disk_encryption

If you want the features I described in my post, I don't think you'll get most of them in a $ 0.50 / GB drive. I had to start looking in the $ 1.00 / GB and up range to get most of these things. Still cannot afford a big one though.

PS. Ron are you still mining crypto coins? I have a 5gh/s running 24/7
it makes a couple pennies a day... :)

Interesting question. The IRS recently came out with a ruling that says cryptocoins are property, and that you have to track the cost and value of each coin from the time it's mined (assuming you report it). This made my last tax return insanely complex. Mainly because of that, I'm not active right now in that space. I still have an interest in it. The laws may change. Also, it appears that GPU's as miners are becoming impractical. So, if I get back into it, I might look into scrypt mining with asics. Here's a place where you can rent space on a mining rig and let someone else worry about the technicalities. I'm sure there are others, both for Litecoin type coins and Bitcoin type coins.

https://www.genesis-mining.com/

Sincerely,

Ron

Here are the details:

01) Data Endurance - This is the rated life of the flash memory
cells.  How many writes can they handle without excessive
degradation?  Sometimes, this data is hard to find.  You want to
look for the TBW or Total Bytes Written spec.  This tells you the
total amount of data you can write to the drive.  There is also a
DWPD or Drive Writes Per Day spec.  This tells you how many times
you can completely write the drive's capacity per day.  Finally,
there is a GB / Day of data written spec.  None of the per day
specs mean anything unless you know how many days are under
consideration.  That number may or may not be the same as the
warranty.  Consumer drives are typically rated for 20 - 30 GB of
writes per day for 3 to 5 years.  Prosumer and entry level data
center drives are usually rated for 50 - 70 GB of writes per day.
Having your system cache on the SSD, using it as a scratch drive or
temp drive, or doing things like continuously rewriting video files
you're editing will use up the quota faster.

So, how much data endurance is enough?  Well, short of running some
detailed analytics software to track your data writing (reading is
not a problem), it's hard to say.  As a consumer, and someone who
pushes pc components hard at times and likes to run them till they
drop, I'd rather err on the side of a prosumer type of device.  One
option is to deploy SSD's which have built in analytics on a test
machine and see how fast the usage quota gets used.

All SSD's have a data endurance spec.  Some manufacturers and spec
sheets may make it hard to find.  If you get a drive that is
dramatically too low, you may find your data at risk much sooner
than you thought.

02) Data Retention - This is a whole different animal.  This spec
is VERY hard to find.  It refers to the time that the drive will
retain your data with power off.  When the drive is new, this can
be many years, even a decade.  However, when the drive has reached
its write endurance limit, this number can drop to as little as 3
months.  You want this number to be as high as possible for as long
as possible.

Tech report has been doing some data endurance testing and data
retention testing on several drives since August 2013.  At this
point, they have stored 600 TB of data (which is about double most
drives' rated lifespan).  The very popular Samsung 840 (which uses
TLC flash, see below) showed some data retention problems after 300
TB of storage and after being left powered down for only a week.
The Samsung also showed over 2000 reallocated sectors after 600 TB
of storage.  For these reasons, I would not buy that drive.

See this article:

http://techreport.com/review/26058/the-ssd-endurance-experiment-data-retention-after-600tb

  One thing this means to me is that I would not want to use an SSD
as an archival backup that might sit on the shelf for months or
years.

03) SLC or MLC, not TLC flash cells -  Every cell of a flash memory
device is essentially a capacitor.  An SLC (Single Level Cell)
device stores one voltage in the capacitor which can represent one
binary bit.  An MLC (Multi Level Cell) device stores 4 voltages (or
possibly 3 plus zero) which can represent two binary bits.  A TLC
(Triple Level Cell, a bit of a misnomer) device stores 8 voltages,
which can represent three binary bits.  So, theoretically, with
TLC, you can store triple the data in the same space on the silicon
chip (more or less).  As such, you can make bigger drives much
cheaper, hence the appeal.  The drives that Tech Report is testing
are all MLC except for the Samsung 840 (which is not the same as
the 840 Pro).  The 840 uses TLC, although Samsung tries to avoid
using that verbiage.  In the test, the 840 was the only one
documented to have retention problems and had FAR more reallocated
sectors than the other drives.  This is why I say to avoid TLC.
SLC is the most reliable technology, but the prices for those
really are sky high.  MLC represents a good compromise between
reliability and price.

04) 5 Year Warranty - Almost every drive you'll find on retail
store shelves, and many at online sellers, will have a 2 - 3 year
warranty.  If you search at higher price levels, around $ 1 / GB,
you'll start to find drives with 5 year warranties.  This is my
preference.  If I'm going to go to the trouble to move all my stuff
to an SSD, or especially if I have to go to the trouble of
splitting things among two drives, or reinstalling the OS; then I
don't want to mess with it too often.  So, I'd go for the longer
warranty.  However, warranties are not simple.  They're often tied
to the data endurance rating, and the warranty will expire sooner
timewise if you exceed the stated quota.  Sometimes, warranties
include tech support for the warranty period.  Sometimes, they
don't.

05) Overprovisioning - The drive makers often include extra space
in the flash memory, compared to the rating of the drive.  So, it
may actually have 128 GB of flash, but the rated capacity may be
120 GB.  So, they've actually reserved 6 - 7% for spare space.
This is a good thing.  The drive uses this space for managing it's
wear leveling activities, garbage collection, trimming unused
sectors, and other housekeeping things.  One very important thing
it uses this for is to provide spare sectors to put into usage when
it reallocates one that's not reading or writing properly.  So, in
the case of the Samsung 840 I mentioned above that had 2000+
reallocated sectors, assuming the drive has that much or more
overprovisioned area, then the usable capacity of the drive will
not have been decreased.  Having to USE that many spare sectors
still makes me nervous.  Even if you can buy a drive without
overprovisioning, or even if you an configure it to eliminate
overprovisioning, I woul dn't do that.  If you're looking at a
drive that has a capacity that's an even multiple of 2, such as 128
GB or 256 GB, as opposed to 120 GB or 240 GB, I would question to
see if it is overprovisioned.

06) TRIM Command - This is a fairly common but not universal (as
far as I know) feature that you definitely want.  If both the OS
and the drive are  TRIM compatible, then the OS can tell the drive
what sectors are no longer needed.  Flash memory has to be erased
in large chunks before it can be reused.  By allowing the OS to
tell the drive what chunks are not needed, it can do this erasure
in the background so that, when a write is required, it will have
erased flash cells ready and waiting.  This means write performance
is improved, particularly if the drive is fairly full.

07) Background Garbage Collection - This allows the drive to
recover unneeded sectors and prepare them for reuse in the
background while the drive is doing other things.  Background
Garbage Collection and TRIM interact, but are not the same.  You
may have TRIM, but not Background Garbage Collection, for example.
TRIM helps the garbage collector know what to zone in on, but the
drive with garbage collection may be able to do more on its own
even when it's not getting TRIM requests from the OS.

08) Power Loss Protection - This is a very important feature.  It
is rare on most low end drives.  SSD's have ram (volatile) caches
just like HDD's.  Sometimes, writes are queued up in the ram prior
to being written to flash.  If the power fails abruptly, those
pending writes can be lost, which can corrupt files and data.
Power Loss Protection provides capacitors on the power bus in the
drive.  This maintains the drive long enough to clear the write
cache and commit all pending writes to flash in the event of a
power failure.

09) SMART - Self-Monitoring, Analysis and Reporting Technology -
Most hard drives have SMART, and most SSD's do too, but not
necessarily all.  This subsystem allows the computer to monitor
various aspects of the health and status of the drive.  Reallocated
sectors is one thing typically monitored by the SMART system, along
with many other parameters.  You definitely want your SSD to have
SMART.

10) Wear Indicator, Life Remaining, Data Written - Most of these
attributes are passed through the SMART system, although some
drives use proprietary software to monitor this.  This,
essentially, tells you in various ways how much of the drive's
endurance rating you've used up and how much drive life is left.  A
very desirable feature.

11) Exotic Diagnostic Data - (That's my term for it.)  Some drives
can monitor and report more exotic and less frequently used, but
still useful, data.  This might include write amplification
statistics, wear leveling statistics, compression statistics, and
temperature, etc.  This data can be valuable to more advanced users
or enterprise users.

12) Wear Leveling - Pretty much all flash storage systems have
this.  It distributes the writes to the drive fairly evenly amongst
all available memory cells.  This prevents any one cell or group of
cells from wearing out substantially before the others.  There are
variations in the way this is accomplished, and some manufacturers
rave on about how their way is better.

13) NCQ - Native Command Queuing - (I'll just copy from Wikipedia
here.)  "NCQ is also used in newer solid-state drives where the
drive encounters latency on the host, rather than the other way
around. For example, Intel's X25-E Extreme solid-state drive uses
NCQ to ensure that the drive has commands to process while the host
system is busy processing CPU tasks.  NCQ also enables the SSD
controller to complete commands concurrently (or partly
concurrently, for example using pipelines) where the internal
organisation of the device enables such processing."

Bottom line, it improves performance, which is always good.

14) Full Drive Encryption - This feature automatically encrypts all
data written to the drive, and decrypts it on the fly when needed.
This is generally 128 bit or 256 bit AES encryption.  This is
similar to what you can do with something like TrueCrypt, but is
faster and more seamless.  If the drive or PC is stolen, and the
password or passphrase or other key is not known to the attacker,
the drive just looks like gibberish.  I don't know, and have not
explored, how this is accessed by the user.  I'm assuming you need
a compatible bios and that it pops up during boot and asks for a
password.  I also don't know if there are some computers that
cannot use an encrypted drive.

15) Extra Robust ECC (Error Correction Code) - All storage drives
include error correction codes in the data stream to compensate for
one or multiple single bit errors in a row due to flaky storage
media.  Sometimes they can correct for more continuous errors,
sometimes less.  Some manufacturers like to rave on about their
more extensive error correction in certain product lines.

16) Compressible and Uncompressible Performance - Almost all, if
not all, SSD's compress data for storage to allow a given amount of
flash cells to do more and thus reduce the cost of the drive.
That's great if your data is compressible, like a text file.
However, it's not so great if the data is not compressible, like
executables, or already compressed data.  Sometimes, the
performance of the drive will greatly decrease when fed
incompressible data.  You want a drive that maintains good
performance regardless.

17) Secure Erase - This feature allows you to completely and
securely erase the SSD in preparation for new usage,
decommissioning, or sale, etc.  This is better than running
something like DBAN on it for two reasons.  First, it's probably
much faster.  Second, if the drive has reallocated sectors, you
won't be able to access them from the normal api and interface.
This function SHOULD include those sectors which are no longer
accessible to the user.  While I know that secure erase has been in
the ATA spec for some time, I don't know if all SSD's support this.
You would want to verify it if it's important to you.

18) Data Center Design - This is a sort of nebulous thing that some
manufacturers and some reviewers rave on about with some products.
An example would be the Intel 730 series of SSD's.  While this
doesn't have a specific meaning, it probably means the drive has
more of the features I've been discussing.  It also probably means
the drive is more expensive.

19) SATA 3 - 6 Gbps Interface - This is an almost universal feature
on SSD's.  The simple reason is that if you make the drive with a
SATA 2 - 3 Gbps Interface, you won't get anywhere near its maximum
performance.  So, when you read data transfer specs in the 550 MBps
range, then you're getting close to saturating the SATA 3 bus.
That's one reason the max data transfer speed on most drives is
similar.  If you put a SATA 3 drive into a SATA 2 or USB 2
enclosure, you'll severely cripple it.

20) Data Cloning Software (4K Sector Support) - You're going to
have to get your data from your old drive to your new one somehow,
probably including the OS, the registry (for Windows), user files,
etc.  Many SSD packages include or include a link to download Data
Cloning Software, such as Acronis.  This is a handy feature.  You
must use newer software which fully supports drives with 4K
sectors.  If you use older software which only supports 512 byte
sectors, and your partitions are misaligned, your performance could
suffer dramatically.  I know this applies to newer hard drives and
believe it applies to SSD's.  There is some debate about it, and I
couldn't prove it while writing this.  I'm assuming SSD's
partitions have to be aligned on 4K boundaries.

21) Physical Size - 2.5" x 7mm - This is obviously not mandatory
for some applications.  However, this size of SSD is the most
versatile and universal.  This form factor allows the drive to fit
in almost anything that will accept a SATA drive.  This includes
almost all notebook / laptop computers.  Some won't take a 9mm
thickness.  Very very small devices might need an mSATA drive or
memory card.  As far as I know, most memory cards don't include
most of these features I've discussed.

22) 3.5" Mounting Adapter - Many drives include this, and it's a
handy addition to allow mounting in desktop drive bays.  Some
desktop cases include drive cages which automatically adapt to
either a 2.5" or 3.5" drive.  I've seen some youtube pc building
videos where someone just velcros the drive to the computer case.
Since it's not mechanical, I guess that could be an option,
although it might create static, which would be bad.

23) Intel or Samsung Preferably - Finally, while I would consider
any major brand name of drive, I've found overwhelmingly in my
reading that Intel and Samsung drives are supposed to be the most
reliable.  So, I would have a potential preference for one of
those, if my other criteria were met.

I hope you find this info helpful.  I know I'm going to be
revisiting this list when I finish saving enough money to buy an
SSD.  Unfortunately, I also know I won't be buying the cheapest
thing in town.

Sincerely,

Ron





--

(PS - If you email me and don't get a quick response, you might want to
call on the phone.  I get about 300 emails per day from alternate energy
mailing lists and such.  I don't always see new email messages very quickly.)

Ron Frazier
770-205-9422 (O)   Leave a message.
linuxdude AT techstarship.com

_______________________________________________
tech-chat mailing list
[email protected]
http://lists.linuxmoose.com/mailman/listinfo/tech-chat

Reply via email to