Re: Thumb drive longevity (was RE: Linux on USB)

2014-04-25 Thread Stephen Partington
I shut of swap/virt mem on SSD's anyhow. then again i also put a large ish
amount of memory in those machines for that purpose.

Good read however. It explains some issues with gaming from an SSD and the
issues inherent with it. and why it works better as an "OS" drive than game
install drive.


On Fri, Apr 25, 2014 at 9:05 AM, Carruth, Rusty wrote:

> Actually, for a single erase block on a thumb drive, its lifetime in
> Erase/Program cycles is more in the range of 100 to 3,000.
>
> Let me back up a bit and explain the structure and technology of flash.
>
> There are 2 kinds of flash (well, becoming 3 - I've not been watching
> TLC): SLC and MLC.  MLC is 'multi-level-cell' which means storing more
> than 1 bit per cell.  Nowadays it has come to mean storing 2 bits in
> each transistor charge well (or whatever they're called).  SLC means
> 'single-level-cell', which means you store one and only 1 bit in each
> possible storage cell.  (The technology here is basically storing some
> number of electrons in a place in the transistor such that you can read
> it back.  A flash chip with 8G bytes of storage would have 8Gx8 storage
> cells for an SLC chip and 8Gx8/2 storage cells for an MLC chip - so you
> get twice the data in the same physical space, but then there are
> tradeoffs we'll get to here in a moment).  (TLC means triple level cell,
> or 3 bits in each cell, but I'm not going to address that here)
>
> The chips themselves are arranged as some number of 'erase blocks', each
> containing some number of 'pages' (with some number of bytes in each
> page).  For one of the SLC chips I know about ISTR that the erase block
> is 1 megabyte, and the page is 8K bytes.  If those numbers are totally
> wrong, they are at least correct to an order of magnitude base 2.
>
> So my example (mythical) 8G chip above would have 8K erase blocks if so
> arranged.  (I should have said 8Gbit not 8GByte, but let's ignore that
> for now and pretend they make 8GByte SLC chips)
>
> Now, the smallest 'thing' you can erase is an erase block.  You are only
> supposed to write once to each page, and you are really supposed to
> write them in order (0 through n, start at zero and increment up by
> one).  If you don't the reliability goes down.
>
> So the process of writing to a flash chip is to erase the erase block
> first, then start writing at page 0 of that erase block (8k at a time!)
> until you reach the end of the block.  Then, if you want to re-write
> something you wrote, you have to write it somewhere else and keep track
> of where you put that thing, and that the copy in the first erase block
> is old and shouldn't be used.  You MAY have to copy the entire erase
> block somewhere else just to write 512 bytes of data!  So to write a
> single 'sector' (512 bytes) to an SSD could result in an entire megabyte
> being written to a flash chip!  (Now, let me first say that a
> well-designed SSD or thumb drive isn't usually THAT bad, but generally a
> single sector written to a flash drive results in more than one sector
> getting written to the flash.  This is called write amplification, and
> I've glossed over MUCH, including how much certain companies (ahem) go
> to to minimize that number).
>
> Ok, back to the whole point. Generally an SLC erase block can be erased
> and written to on the order of 100,000 to 300,000 times before its
> failure rate goes up (or it fails completely).  MLC (2-bits per cell) is
> on the order of 500 to 3,000.  Your thumb drives are made of this. Most
> consumer-grade SSDs are also. (A side note - TLC (THREE bits per cell!)
> is even worse on this)
>
> I should mention in passing 'overprovisioning'.  This is where some
> percentage (often 7%) of the flash is kept back from the available
> user-visible space.  For example, if a drive had 128GB worth of flash,
> only 100GB would be available when you look at the 'fdisk' output for
> that drive.  This allows the drive to last longer and do 'wear
> levelling'.  (You obviously have to have at least 1 erase block that you
> don't assign to the user because you have to have somewhere to write
> this new data which replaces the old data)
>
> Oh, by the way, with today's geometry I understand that the difference
> between a zero and a one (SLC!) is around 100 ELECTRONS!  It is 1/4 of
> that for MLC (since there are 4 values to be stored in the cell, not
> just 2).
>
> So at one level (and with many assumptions and simplifications such that
> I could be off by as much as a factor of 4, but not usually in your
> favor), if you have written 1,000 times the disk capacity to the drive
> (so if you have a 1GB drive and you write 1,000GB to it (regardless of
> location)) then you have probably come close to having performed 1,000
> program and erase cycles on each erase block.
>
> The 'good news' is that that 300,000 number for SLC (and 3,000 for MLC)
> is not where ALL the erase blocks go bad.  You've got some life left in
> those chips, but you are on the way to to

Thumb drive longevity (was RE: Linux on USB)

2014-04-25 Thread Carruth, Rusty
Actually, for a single erase block on a thumb drive, its lifetime in
Erase/Program cycles is more in the range of 100 to 3,000.  

Let me back up a bit and explain the structure and technology of flash.

There are 2 kinds of flash (well, becoming 3 - I've not been watching
TLC): SLC and MLC.  MLC is 'multi-level-cell' which means storing more
than 1 bit per cell.  Nowadays it has come to mean storing 2 bits in
each transistor charge well (or whatever they're called).  SLC means
'single-level-cell', which means you store one and only 1 bit in each
possible storage cell.  (The technology here is basically storing some
number of electrons in a place in the transistor such that you can read
it back.  A flash chip with 8G bytes of storage would have 8Gx8 storage
cells for an SLC chip and 8Gx8/2 storage cells for an MLC chip - so you
get twice the data in the same physical space, but then there are
tradeoffs we'll get to here in a moment).  (TLC means triple level cell,
or 3 bits in each cell, but I'm not going to address that here)

The chips themselves are arranged as some number of 'erase blocks', each
containing some number of 'pages' (with some number of bytes in each
page).  For one of the SLC chips I know about ISTR that the erase block
is 1 megabyte, and the page is 8K bytes.  If those numbers are totally
wrong, they are at least correct to an order of magnitude base 2.

So my example (mythical) 8G chip above would have 8K erase blocks if so
arranged.  (I should have said 8Gbit not 8GByte, but let's ignore that
for now and pretend they make 8GByte SLC chips)

Now, the smallest 'thing' you can erase is an erase block.  You are only
supposed to write once to each page, and you are really supposed to
write them in order (0 through n, start at zero and increment up by
one).  If you don't the reliability goes down.

So the process of writing to a flash chip is to erase the erase block
first, then start writing at page 0 of that erase block (8k at a time!)
until you reach the end of the block.  Then, if you want to re-write
something you wrote, you have to write it somewhere else and keep track
of where you put that thing, and that the copy in the first erase block
is old and shouldn't be used.  You MAY have to copy the entire erase
block somewhere else just to write 512 bytes of data!  So to write a
single 'sector' (512 bytes) to an SSD could result in an entire megabyte
being written to a flash chip!  (Now, let me first say that a
well-designed SSD or thumb drive isn't usually THAT bad, but generally a
single sector written to a flash drive results in more than one sector
getting written to the flash.  This is called write amplification, and
I've glossed over MUCH, including how much certain companies (ahem) go
to to minimize that number).

Ok, back to the whole point. Generally an SLC erase block can be erased
and written to on the order of 100,000 to 300,000 times before its
failure rate goes up (or it fails completely).  MLC (2-bits per cell) is
on the order of 500 to 3,000.  Your thumb drives are made of this. Most
consumer-grade SSDs are also. (A side note - TLC (THREE bits per cell!)
is even worse on this)

I should mention in passing 'overprovisioning'.  This is where some
percentage (often 7%) of the flash is kept back from the available
user-visible space.  For example, if a drive had 128GB worth of flash,
only 100GB would be available when you look at the 'fdisk' output for
that drive.  This allows the drive to last longer and do 'wear
levelling'.  (You obviously have to have at least 1 erase block that you
don't assign to the user because you have to have somewhere to write
this new data which replaces the old data)

Oh, by the way, with today's geometry I understand that the difference
between a zero and a one (SLC!) is around 100 ELECTRONS!  It is 1/4 of
that for MLC (since there are 4 values to be stored in the cell, not
just 2).

So at one level (and with many assumptions and simplifications such that
I could be off by as much as a factor of 4, but not usually in your
favor), if you have written 1,000 times the disk capacity to the drive
(so if you have a 1GB drive and you write 1,000GB to it (regardless of
location)) then you have probably come close to having performed 1,000
program and erase cycles on each erase block.

The 'good news' is that that 300,000 number for SLC (and 3,000 for MLC)
is not where ALL the erase blocks go bad.  You've got some life left in
those chips, but you are on the way to total failure at some point in
the future.

Scary stuff, but yes, I do use flash drives (thumb drives) all the time.
However, as they say:

"Back up early, back up often!"


Disclaimer: I work for a company that makes SSDs, so I do know a little
bit about flash ;-), but this is NOT professional advice!  If you want
professional advice go see a shrink ;-)  Also, certain simplifications
have been made, and I've skipped some important details if you were to
be making your own flash drive!