Re: Recommended SSDs and 4-bay internal dock
On Wed, 11 Jan 2023 18:20:05 +0100 Linux-Fan wrote: > IBM says that for enterprise drives (typically higher quality than > consumer- grade drives) only three months of data retention are > guaranteed at 40°C. And: This article is from 2021, too... Well, it turns out I was in a good position to do an experiment. I had a laptop die a year ago. The last file date in the user home directory is Dec 24 2021. I just did an SSD-ectomy and plugged it in to my desktop. The three ext4 partitions all fscked well, and mounted correctly. I also ran a script that checks files against checksum files. It reported no errors. Two of the three partitions are LUKS encrypted. ntfsfix reported one partition with a bad alternate boot sector, and one clean. gsmartcontrol reports the drive to be healthy. It passed an extended self-test. The drive is a Samsung SSD 840 EVO 250GB. One drive does not a thorough study make, but I wouldn't be too worried. -- Does anybody read signatures any more? https://charlescurley.com https://charlescurley.com/blog/ pgpanjCUlYVua.pgp Description: OpenPGP digital signature
Re: Periodic refresh (or rwrite?) of data on an SSD (was: Re: Recommended SSDs and 4-bay internal dock)
On Sat, 14 Jan 2023, at 15:49, Dan Ritter wrote: > Let's separate the problem into three cases: > > - I recently deleted some files, I want them back. (Same as "I > recently changed some files, I want the old versions.") > > - I have lost a filesystem for some reason, I want it restored. > > - I need to have archival copies of some data which I probably > will never access, but just in case... > > > The deleted/changed files problem is best solved, if plausible, > with a snapshotting filesystem like ZFS, with automatic > snapshots and automatic deletion of sufficiently old snapshots. Although this is a debian list, not all my data is on systems that support such FSes. Eg on Windows ... There, I keep all the data where I want access to prior versions of files in Dropbox; I have a year's worth of old versions of all those files. (But finding the right one to reinstate is clumsy.) I also sometimes run a program that monitors selected filetypes and filenames within a specific directory and every time one of the matching files changes makes a date & time-time suffixed filename copy of that just-changed file. So if I was editing file xyz.pqr in a GUI appplication, every time I do a File->Save from within it as well as the xyz.pqr filebeing written to disk, a new file is created, named xyz (svd@20120716-190723).pqr containing a copy of the just-saved file. I first started using this when I was trying to reproduce, for the programmer, some file-corrupting tendencies in his application and wanted to make saved copies of the file after every change I made to it in the GUI. I kept detailed notes of exactly what I did corresponding to each "(svd@20120716-190723)" value. The programmer wasn't appreciative. He said "no other user complains as much as you do", rather than saying "thank you very much for providing detailed steps to reproduce each of these problems". Bah! > The restore-a-filesystem problem is best solved with a complete > filesystem copy to a similarly sized disk. SSDs are nice and > fast, but you may not actually need that speed if you don't have > to do a restore very often. ZFS send or rsync are good tools > here, or borg if you have special requirements. It's not so much the speed that attracts me to SSDs as the lack of moving parts and their resistance to shock. > The archive problem is best solved with spinning disks, which > are fast enough for most cases, much better priced per capacity, > and have well-understood stability over long periods of > unpowered time. Speed is not a colossal issue - backups can always run while I eat or sleep or whatever. Price is not that much of an issue either, as I know the hard way that angst & the time cost spent recovering lost data rapidly outweigh the minor capital cost of more backup devices. The long-term stability of spinning rust is certainly becoming a lot more important to me. But I already plan to store future backups on both types of disk. -- Jeremy Nicoll - my opinions are my own.
Re: Periodic refresh (or rwrite?) of data on an SSD (was: Re: Recommended SSDs and 4-bay internal dock)
Jeremy Nicoll wrote: > On Fri, 13 Jan 2023, at 15:33, rhkra...@gmail.com wrote: > > The whole issue makes me wonder if, say, I should plan on having several > SSDs for each set of backup data (I mean separately from the common- > sense approach of having more than one copy of anything anyway). Then > every n weeks, delete the data from the least-recently written drive and > copy a fresh copy (from the most recently written drive) onto that one, & > verify that every file copied has the same hash as its original. (I suspect > I'd want to keep lists of file hashes anyway, as a way of detecting when > any backups start to go bad.) > > I'm not sure that that was clear. What I mean is that if I intended to keep > a backup of a driveful of data, I might choose to have, say, this week's > copy, last week's, and the week before. So apart from the original disk > I'd have 3 other backup drives. Let's separate the problem into three cases: - I recently deleted some files, I want them back. (Same as "I recently changed some files, I want the old versions.") - I have lost a filesystem for some reason, I want it restored. - I need to have archival copies of some data which I probably will never access, but just in case... The deleted/changed files problem is best solved, if plausible, with a snapshotting filesystem like ZFS, with automatic snapshots and automatic deletion of sufficiently old snapshots. The restore-a-filesystem problem is best solved with a complete filesystem copy to a similarly sized disk. SSDs are nice and fast, but you may not actually need that speed if you don't have to do a restore very often. ZFS send or rsync are good tools here, or borg if you have special requirements. The archive problem is best solved with spinning disks, which are fast enough for most cases, much better priced per capacity, and have well-understood stability over long periods of unpowered time. Grabbing a copy from your restore-a-filesystem system every so often might be the way to go. In any case, think about backing up filesystems rather than disks. Sometimes they are the same, but not very often. -dsr-
Re: Periodic refresh (or rwrite?) of data on an SSD (was: Re: Recommended SSDs and 4-bay internal dock)
On Fri, 13 Jan 2023, at 15:33, rhkra...@gmail.com wrote: > I've read some of that article, or, I guess, really the abstract and the > section labeled "Content" on that page: > > https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-shutdown Is there a longer article? I have the impression that the Abstract & Content on that page is all there is. There seems to me to be some ambiguity about the way they write about "40C" which - I presume - is ambient temperature. Are they saying that these drives are rated to be used in temperatures not exceeding 40 degrees C (ie that if the temp is even higher the situation is likely (far?) worse than what's described here? That is, should the sentence "The JEDEC spec for Enterprise SSD drives requires that the drives retain data for a minimum of 3 months at 40C." be read as "The JEDEC spec for Enterprise SSD drives requires that the drives retain data for a minimum of 3 months at /temperatures of up to but not exceeding/ 40C." because this is also ambiguous "This means that after 3 months of a system being powered off in an environment that is at 40C or less, there is a potential of data loss ..." Either you can read that as implying the "40C or less" is part of the cause of data loss, or that it means "that after 3 months of a system being powered off in an environment that is /working as designed/ at 40C or less, there is a potential of data loss" that is, that it's the lack of power that's the cause, not the temperature. >From a chilly UK standpoint, 40 degrees C seems very high. I wonder if data retention is better or worse at - say - ambient temps of eg 15-20 degrees? I wonder how much worse home-user SSDs are than these Enterprise-rated ones. The whole issue makes me wonder if, say, I should plan on having several SSDs for each set of backup data (I mean separately from the common- sense approach of having more than one copy of anything anyway). Then every n weeks, delete the data from the least-recently written drive and copy a fresh copy (from the most recently written drive) onto that one, & verify that every file copied has the same hash as its original. (I suspect I'd want to keep lists of file hashes anyway, as a way of detecting when any backups start to go bad.) I'm not sure that that was clear. What I mean is that if I intended to keep a backup of a driveful of data, I might choose to have, say, this week's copy, last week's, and the week before. So apart from the original disk I'd have 3 other backup drives. Then for each of those there'd be two other drives in use, so 9 in all... -- Jeremy Nicoll - my opinions are my own.
Re: Periodic refresh (or rwrite?) of data on an SSD (was: Re: Recommended SSDs and 4-bay internal dock)
rhkra...@gmail.com writes: On Wednesday, January 11, 2023 12:20:05 PM Linux-Fan wrote: > > Or does one need to read every byte, allocated or not? > > AFAIK one needs to _power_ the device every once in a while and keep power > connected for some time. Then, the controller can dos all the necessary > actions in the background. [...] > This entry seems to be rather pessimistic: > https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-sh > utdown I've read some of that article, or, I guess, really the abstract and the section labeled "Content" on that page: https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended- shutdown I see the statement: "A system (and its enclosed drives) should be powered up at least 2 weeks after 2 months of system power off. If a drive has an error indicating it is at end of life we recommend not powering off the system for extended periods of time.", and the first quoted paragraph in this email reiterates the need to power up occasionally and leave connected for some time (so that the controller can do all the necessary actions in the background). I assume that they are talking about the hardwired controller built into the drive, thus there is no particular need to power it up from an OS that recognizes it but simply something that powers it somehow? Actually, since the IBM article is about some sort of storage system, it becomes hard to tell what IBM mean with the "controller": The two weeks are quite long a period i.e. they expect some technician to swap SSDs in the storage array once in a while and in there, they are powered 24/7. For consumer SSDs, I understand the controller to be indeed one of the chips on the SSD i.e. outside the realm of the OSes control. Also, no consumer SSD manufacturer advises their customers to power the SSDs two weeks straight :) Can anyone shed more light on what happens during that two weeks -- is the data somehow "refreshed" (in place), or rewritten somewhere else on the drive, or ??? (Perhaps that is discussed in the complete article of which this appears to be just the abstract??) For the IBM case I expect that information to be proprietary :) In fact, there seems to be curiously little information available on the topic at all. Most people (including credible sources like e.g. the Kingston Support) suggest that all data should be read from the flash for the controller to refresh it: https://reboot.pro/index.php?s=234d281f8a9f18ba7b36f5e98890bd2f=13791#entry121995 It is probably not the entire truth though, because if I understand some JEDEC slides correcly, thery only talk about power-off where I assume that everything non-power-off must not count towards the data retention time including not reading the data? (cf. slides 26f of https://www.jedec.org/sites/default/files/Alvin_Cox%20[Compatibility%20Mode]_0.pdf) Official manufacturer documents are pretty opaque regarding the issue: https://www.kingston.com/en/blog/pc-performance/ssd-garbage-collection-trim-explained https://semiconductor.samsung.com/resources/white-paper/Samsung_SSD_White_Paper.pdf (relevant chapter is CH04 with pp. 17ff) [...] Maybe someone with advanced Interenet searching capabilities can bring up more relevant documents about the subject :) HTH Linux-Fan öö pgpoR_XL95c31.pgp Description: PGP signature
Periodic refresh (or rwrite?) of data on an SSD (was: Re: Recommended SSDs and 4-bay internal dock)
On Wednesday, January 11, 2023 12:20:05 PM Linux-Fan wrote: > > Or does one need to read every byte, allocated or not? > > AFAIK one needs to _power_ the device every once in a while and keep power > connected for some time. Then, the controller can dos all the necessary > actions in the background. ... > Here is an article from 2021 that shows some typical numbersas I > remembered them. I do not know anything about this specific source's > credibility, though: > https://www.virtium.com/knowledge-base/ssd-data-retention/ > > This entry seems to be rather pessimistic: > https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-sh > utdown I've read some of that article, or, I guess, really the abstract and the section labeled "Content" on that page: https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended- shutdown I see the statement: "A system (and its enclosed drives) should be powered up at least 2 weeks after 2 months of system power off. If a drive has an error indicating it is at end of life we recommend not powering off the system for extended periods of time.", and the first quoted paragraph in this email reiterates the need to power up occasionally and leave connected for some time (so that the controller can do all the necessary actions in the background). I assume that they are talking about the hardwired controller built into the drive, thus there is no particular need to power it up from an OS that recognizes it but simply something that powers it somehow? Can anyone shed more light on what happens during that two weeks -- is the data somehow "refreshed" (in place), or rewritten somewhere else on the drive, or ??? (Perhaps that is discussed in the complete article of which this appears to be just the abstract??) Thanks! -- rhk (sig revised 20221206) If you reply: snip, snip, and snip again; leave attributions; avoid HTML; avoid top posting; and keep it "on list". (Oxford comma (and semi-colon) included at no charge.) If you revise the topic, change the Subject: line. If you change the topic, start a new thread. Writing is often meant for others to read and understand (legal documents excepted?) -- make it easier for your reader by various means, including liberal use of whitespace (short paragraphs, separated by whitespace / blank lines) and minimal use of (obscure?) jargon, abbreviations, acronyms, and references. If someone has already responded to a question, decide whether any response you add will be helpful or not ... A picture is worth a thousand words. A video (or "audio"): not so much -- divide by 10 for each minute of video (or audio) or create a transcript and edit it to 10% of the original. A speaker who uses ahhs, ums, or such may have a real physical or mental disability, or may be showing disrespect for his listeners by not properly preparing in advance and thinking before speaking. (Remember Cicero who did not have enough time to write a short missive.) (That speaker might have been "trained" to do this by being interrupted often if he pauses.) A radio (or TV) station which broadcasts speakers with high pitched voices (or very low pitched / gravelly voices) (which older people might not be able to hear properly) disrespects its listeners. Likewise if it broadcasts extraneous or disturbing sounds (like gunfire or crying), or broadcasts speakers using their native language (with or without an overdubbed translation). A person who writes a sig this long probably has issues and disrespects (and offends) a large number of readers. ;-) '
Re: Recommended SSDs and 4-bay internal dock
On 1/11/23 06:58, Tom Browder wrote: I plan to install a 4-bay, hot swappable SSD dock to replace the existing DVD in my only 5.5" externally accesible bay. To fill it, I will get up to four 2.5 inch SSDs of 1 Tb: MX500 by Crucial. My plan is to use the SSDs for backup, but not in a RAID configuration. I would appreciate opinions on my choice of SSD, as well as which brand of dock is recommended: ICY DOCK versus StarTech. On Amazon, the ICY DOCK mentions a lithium battery but the StarTech doesn't, and I'm not sure why it's needed. Thanks. -Tom I have been using StarTech drive racks for many years. Out of two dozen or so 3.5" HDD racks over the past ~20 years, perhaps a half dozen fans wore out and one PCB died. All the other issues were drives, cables, HBA's, and/or PEBKAC. As other readers have mentioned, failure modes of HDD's are reasonably well understood; as are recovery options. AIUI solid-state devices are all-or-nothing -- they work and then they become bricks. I use internal 3.5" HDD's in ZFS mirrors for live data, backups, archives, and images. I also use single 3.5" HDD's in StarTech DRW150SATBK racks for near-site and off-site duplicates. The speed of SSD's is appealing for remote site work, but I would want two SSD's for safety. David
Re: Recommended SSDs and 4-bay internal dock
On 11/01/2023 14:58, Tom Browder wrote: I plan to install a 4-bay, hot swappable SSD dock to replace the existing DVD in my only 5.5" externally accesible bay. To fill it, I will get up to four 2.5 inch SSDs of 1 Tb: MX500 by Crucial. My plan is to use the SSDs for backup, but not in a RAID configuration. I would appreciate opinions on my choice of SSD, as well as which brand of dock is recommended: ICY DOCK versus StarTech. On Amazon, the ICY DOCK mentions a lithium battery but the StarTech doesn't, and I'm not sure why it's needed. I have exactly the same SSDs installed. Working in Btrfs Raid1 mode as /home in my desktop. SMART data: Device Model: CT1000MX500SSD1 (4096 bytes sector) Power On: 1099 hours Power Cycles: 213 (5 hours/cycle) Temperature: 29 °C Unused Reserve NAND Blocks: 56 Lifetime Used: 2% Average Block-Erase Count: 33 Total Written: 71 TiB Device Model: CT1000MX500SSD1 (4096 bytes sector) Power On: 1095 hours Power Cycles: 214 (5 hours/cycle) Temperature: 29 °C Unused Reserve NAND Blocks: 61 Lifetime Used: 2% Average Block-Erase Count: 28 Total Written: 67 TiB All good. -- With kindest regards, Piotr. ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ Debian - The universal operating system ⢿⡄⠘⠷⠚⠋⠀ https://www.debian.org/ ⠈⠳⣄
Re: Recommended SSDs and 4-bay internal dock
Jeremy Nicoll writes: On Wed, 11 Jan 2023, at 14:58, Tom Browder wrote: > I plan to install a 4-bay, hot swappable SSD dock to replace the existing > DVD in my only 5.5" externally accesible bay. To fill it, I will get up to > four 2.5 inch SSDs of 1 Tb: MX500 by Crucial. My plan is to use the SSDs > for backup, but not in a RAID configuration. I can't advise on choice of dock etc, but I'm interested in the side issue of how long an unpowered SSD can be assumed still to be holding the data written to it. Does one need (just) to mount the drive once in a while? (How often?) Or does one need (say) to read every file on the drive once, so that the SSD controller can assess whether any data needs to be moved? Or does one need to read every byte, allocated or not? AFAIK one needs to _power_ the device every once in a while and keep power connected for some time. Then, the controller can dos all the necessary actions in the background. A long time ago, companies claimed data retention for 10 years (that was: for SLC drives!). The latest figure that I am aware of was 1 year (maybe for TLC?). I think the trend is that manymanufacturers do not publish any data retention times for consumer drives (newly QLC) anymore. One can only guess or measurea. For backup purposes, I believe the advantage of SSDs over HDDs is mostly that they are shock resistant. If this is of no concern, I'd prefer to backup to HDDs instead of SSDs because of the data retention issue and in general a higher chance of rescuing data from the drive in event of failure. Here is an article from 2021 that shows some typical numbersas I remembered them. I do not know anything about this specific source's credibility, though: https://www.virtium.com/knowledge-base/ssd-data-retention/ This entry seems to be rather pessimistic: https://www.ibm.com/support/pages/potential-ssd-data-loss-after-extended-shutdown IBM says that for enterprise drives (typically higher quality than consumer- grade drives) only three months of data retention are guaranteed at 40°C. And: This article is from 2021, too... YMMV Linux-Fan öö pgpjiKjHb2kfb.pgp Description: PGP signature
Re: Recommended SSDs and 4-bay internal dock
On Wed, Jan 11, 2023 at 09:10 Dan Ritter wrote: … > I have lots of MX500 1TB SSDs in service, and they are perfectly > fine, largely unremarkable devices … > Thanks, Dan—good to know. > -Tom
Re: Recommended SSDs and 4-bay internal dock
On Wed, 11 Jan 2023, at 14:58, Tom Browder wrote: > I plan to install a 4-bay, hot swappable SSD dock to replace the existing > DVD in my only 5.5" externally accesible bay. To fill it, I will get up to > four 2.5 inch SSDs of 1 Tb: MX500 by Crucial. My plan is to use the SSDs > for backup, but not in a RAID configuration. I can't advise on choice of dock etc, but I'm interested in the side issue of how long an unpowered SSD can be assumed still to be holding the data written to it. Does one need (just) to mount the drive once in a while? (How often?) Or does one need (say) to read every file on the drive once, so that the SSD controller can assess whether any data needs to be moved? Or does one need to read every byte, allocated or not? -- Jeremy Nicoll - my opinions are my own.
Re: Recommended SSDs and 4-bay internal dock
Tom Browder wrote: > I plan to install a 4-bay, hot swappable SSD dock to replace the existing > DVD in my only 5.5" externally accesible bay. To fill it, I will get up to > four 2.5 inch SSDs of 1 Tb: MX500 by Crucial. My plan is to use the SSDs > for backup, but not in a RAID configuration. > > I would appreciate opinions on my choice of SSD, as well as which brand of > dock is recommended: ICY DOCK versus StarTech. On Amazon, the ICY DOCK > mentions a lithium battery but the StarTech doesn't, and I'm not sure why > it's needed. I have lots of MX500 1TB SSDs in service, and they are perfectly fine, largely unremarkable devices. I have some ICY DOCK and Startech disk-related things; not very many, and not the specific things you are looking for here. In my experience, they are both perfectly fine. -dsr-
