Re: [gentoo-user] which linux RAID setup to choose?
Am 03.05.2020 um 07:44 schrieb Caveman Al Toraboran: * RAIDs 4 to 6: fails to satisfy point (3) since they are stuck with a fixed tolerance towards failing disks (i.e. RAIDs 4 and 5 tolerate only 1 disk failure, and RAID 6 tolerates only 2). As far as I remember there can be spare drives / partitions which will replace a failed one if needed. But this does not help if drives / partitions fail at the same moment. Under normal conditions spares will rise the number of drives which can fail. Nothing you asked but I had very bad experience with drives which spin down by themselves to save energy (mostly titled green or so). Also there has been some talk about SMR https://hardware.slashdot.org/story/20/04/19/0432229/storage-vendors-are-quietly-slipping-smr-disks-into-consumer-hard-drives
Re: [gentoo-user] which linux RAID setup to choose?
On 03/05/20 06:44, Caveman Al Toraboran wrote: > hi - i'm to setup my 1st RAID, and i'd appreciate > if any of you volunteers some time to share your > valuable experience on this subject. > > my scenario > --- > > 0. i don't boot from the RAID. > > 1. read is as important as write. i don't >have any application-specific scenario that >makes me somehow favor one over another. >so RAIDs that speed up the read (or write) >while significantly harming the write (or >read) is not welcome. > > 2. replacing failed disks may take a week or >two. so, i guess that i may have several >disks fail one after another in the 1-2 >weeks (specially if they were bought >about the same time). > > 3. i would like to be able to grow the RAID's >total space (as needed), and increase its >reliability (i.e. duplicates/partities) as >needed. > >e.g. suppose that i got a 2TB RAID that >tolerates 1 disk failure. i'd like to, at >some point, to have the following options: > > * only increase the total space (e.g. >make it 3TB), without increasing >failure toleration (so 2 disk failure >would result in data loss). > > * or, only increase the failure tolerance >(e.g. such that 2 disks failure would >not lead to data loss), without >increasing the total space (e.g. space >remains 2TB). > > * or, increase, both, the space and the >failure tolerance at the same time. > > 4. only interested in software RAID. > > my thought > -- > > i think these are not suitable: > > * RAID 0: fails to satisfy point (3). > > * RAID 1: fails to satisfy points (1) and (3). > > * RAIDs 4 to 6: fails to satisfy point (3) > since they are stuck with a fixed tolerance > towards failing disks (i.e. RAIDs 4 and 5 > tolerate only 1 disk failure, and RAID 6 > tolerates only 2). > > > this leaves me with RAID 10, with the "far" > layout. e.g. --layout=n2 would tolerate the > failure of two disks, --layout=n3 three, etc. or > is it? (i'm not sure). > > my questions > > > Q1: which RAID setup would you recommend? I'd recommend having a spare in the array. That way, a single failure would not affect redundancy at all. You can then replace the spare at your leisure. If you want to grow the array, I'd also suggest "raid 5 + spare". That's probably better than 6 for writing. but 6 is better than 5 for redundancy. Look at having a journal - that could speed up write speed for raid 6. > > Q2: how would the total number of disks in a > RAID10 setup affect the tolerance towards > the failing disks? > Sadly, it doesn't. If you have two copies, losing two disks COULD take out your raid. > if the total number of disks is even, then > it is easy to see how this is equivalent > to the classical RAID 1+0 as shown in > md(4), where any disk failure is tolerated > for as long as each RAID1 group has 1 disk > failure only. That's a gamble ... > > so, we get the following combinations of > disk failures that, if happen, we won't > lose any data: > > RAID0 > --^-- > RAID1 RAID1 > --^-- --^-- > F . . . < cases with > . F . . < single disk > . . F . < failures > . . . F < > > F . . F < cases with > . F F . < two disk > . F . F < failures > F . F . < > . F F . < > > this gives us 4+5=9 possible disk failure > scenarious where we can survive it without > any data loss. > > but, when the number of disks is odd, then > written bytes and their duplicates will > start wrap around, and it is difficult for > me to intuitively see how would this > affect the total number of scenarious > where i will survive a disk failure. > > Q3: what are the future growth/shrinkage > options for a RAID10 setup? e.g. with > respect to these: > > 1. read/write speed. iirc far is good for speed. > 2. tolerance guarantee towards failing >disks. Guarantees? If you have two mirrors. the guarantee is just ONE disk. Yes you can gamble on losing more. > 3. total available space. iirc you can NOT grow the far layout. > > rgrds, > cm. > You have looked at the wiki - yes I know I push it regularly :-) https://raid.wiki.kernel.org/index.php/Linux_Raid Cheers, Wol
Re: [gentoo-user] which linux RAID setup to choose?
On 03/05/20 08:53, hitachi303 wrote: > Nothing you asked but I had very bad experience with drives which spin > down by themselves to save energy (mostly titled green or so). Good catch! For anything above raid 1, MAKE SURE your drives support SCT/ERC. For example, Seagate Barracudas are very popular desktop drives, but I guess maybe HALF of the emails asking for help recovering an array on the raid list involve them dying ... (I've got two :-( but my new system - when I get it running - has ironwolves instead.) Cheers, Wol
[gentoo-user] which linux RAID setup to choose?
hi - i'm to setup my 1st RAID, and i'd appreciate if any of you volunteers some time to share your valuable experience on this subject. my scenario --- 0. i don't boot from the RAID. 1. read is as important as write. i don't have any application-specific scenario that makes me somehow favor one over another. so RAIDs that speed up the read (or write) while significantly harming the write (or read) is not welcome. 2. replacing failed disks may take a week or two. so, i guess that i may have several disks fail one after another in the 1-2 weeks (specially if they were bought about the same time). 3. i would like to be able to grow the RAID's total space (as needed), and increase its reliability (i.e. duplicates/partities) as needed. e.g. suppose that i got a 2TB RAID that tolerates 1 disk failure. i'd like to, at some point, to have the following options: * only increase the total space (e.g. make it 3TB), without increasing failure toleration (so 2 disk failure would result in data loss). * or, only increase the failure tolerance (e.g. such that 2 disks failure would not lead to data loss), without increasing the total space (e.g. space remains 2TB). * or, increase, both, the space and the failure tolerance at the same time. 4. only interested in software RAID. my thought -- i think these are not suitable: * RAID 0: fails to satisfy point (3). * RAID 1: fails to satisfy points (1) and (3). * RAIDs 4 to 6: fails to satisfy point (3) since they are stuck with a fixed tolerance towards failing disks (i.e. RAIDs 4 and 5 tolerate only 1 disk failure, and RAID 6 tolerates only 2). this leaves me with RAID 10, with the "far" layout. e.g. --layout=n2 would tolerate the failure of two disks, --layout=n3 three, etc. or is it? (i'm not sure). my questions Q1: which RAID setup would you recommend? Q2: how would the total number of disks in a RAID10 setup affect the tolerance towards the failing disks? if the total number of disks is even, then it is easy to see how this is equivalent to the classical RAID 1+0 as shown in md(4), where any disk failure is tolerated for as long as each RAID1 group has 1 disk failure only. so, we get the following combinations of disk failures that, if happen, we won't lose any data: RAID0 --^-- RAID1 RAID1 --^-- --^-- F . . . < cases with . F . . < single disk . . F . < failures . . . F < F . . F < cases with . F F . < two disk . F . F < failures F . F . < . F F . < this gives us 4+5=9 possible disk failure scenarious where we can survive it without any data loss. but, when the number of disks is odd, then written bytes and their duplicates will start wrap around, and it is difficult for me to intuitively see how would this affect the total number of scenarious where i will survive a disk failure. Q3: what are the future growth/shrinkage options for a RAID10 setup? e.g. with respect to these: 1. read/write speed. 2. tolerance guarantee towards failing disks. 3. total available space. rgrds, cm.
Re: [gentoo-user] which linux RAID setup to choose?
On Sunday, May 3, 2020 1:14 PM, Wols Lists wrote: > > Q3: what are the future growth/shrinkage > > options for a RAID10 setup? e.g. with > > respect to these: > > > > 1. read/write speed. > > > > iirc far is good for speed. > > > 2. tolerance guarantee towards failing > >disks. > > > > Guarantees? If you have two mirrors. the guarantee is just ONE disk. Yes > you can gamble on losing more. > > > 3. total available space. > > > > iirc you can NOT grow the far layout. sorry, typo, i meant "near" (the command was right though --layout=n2)
Re: [gentoo-user] which linux RAID setup to choose?
On 5/3/20 1:44 AM, Caveman Al Toraboran wrote: [snip]... so, we get the following combinations of disk failures that, if happen, we won't lose any data: RAID0 --^-- RAID1 RAID1 --^-- --^-- F . . . < cases with . F . . < single disk . . F . < failures . . . F < F . . F < cases with . F F . < two disk . F . F < failures F . F . < . F F . < this gives us 4+5=9 possible disk failure scenarious where we can survive it without any data loss. Minor point - you have one duplicate line there ". f f ." which is the second and last line of the second group. No effect on anything else in the discussion. Trying to help thinking about odd numbers of disks, if you are still allowing only one disk to fail, then you can think about mirroring half disks, so each disk has half of it mirrored to a different disk, instead of drives always being mirrored in pairs.
[gentoo-user] Re: Problem understanding "eix"
Martin, On Thursday, 2020-04-30 17:20:08 -, you wrote: > ... > >>=app-crypt/tpm2-tss-2.2.3-r1 ~amd64 > > Ah! That explains it. > > > But this only means that I accept an unstable package here, not that > > these versions are regarded stable. > > It is stabe according to the local configuration on your system. I STRONGLY beg to disagree! The "~amd64" notation is used to ACCEPT a package even though it is (still) classified as UNSTABLE. And yes, the "{isunstable}" property does in fact still return "True". Phrases like the one above only try to conver-up sloppy programming, sloppy document- ation, or both. If "{!isstable}" isn't equivalent to "{isunstable}", there's a severe logical problem involved. > What you look for is perhaps {wasstable} which returns the value > according to the default configuration. That property is indeed mentioned in "man eix" but never explained, like plenty of other properties, too. Besides, in my book "was stable" sort of means "is no longer stable", but again, that's most probaly not what "{wasstable}" means in Eix-speak. But apart from that: you helped me solve my problem, and I thank you for that :-) Sincerely, Rainer
[gentoo-user] Re: Problem understanding "eix"
Dr Rainer Woitok wrote: > I STRONGLY beg to disagree! The "~amd64" notation is used to ACCEPT a > package even though it is (still) classified as UNSTABLE. This is package-manager terminology which has much less states since a package manager needs no fine distinctions about the reasons of accepting or rejecting a package and which of these reasons are caused by your local configuration. In eix there's several configurations: The default (repository) one and the local one (actually even another one with a local profile override). A lot of states (stable, masked, package-masked, etc) can change between these configurations. You can of course invent a new term for each of the several dozens or more possible combination of states, but it is much simpler and more natural to allow all checks separately and use terms for the individual properties which are similar to that of portage. For the optical output, eix displays less information which are more similar to portage. > If "{!isstable}" isn't equivalent to "{isunstable}", > there's a severe logical problem involved. There would be a severe information problem if there were just a few such states and the natural term "unstable" with the analogous "alienunstable" would have been reserved for a mere negation. > Besides, in my book "was stable" sort of means "is no longer stable" It means it was stable before your local overrides. It may or not be stable after your local overrides. Analogously for alienstable, alienunstable, masked, etc.
Re: [gentoo-user] Nvidia driver plus kernel info questions
Dale wrote: > tu...@posteo.de wrote: >> >> Hi Dale, >> >> I am now running kernel 5.6.9 (vanilla) and nvidia-driver 440.82-r3. >> >> Again, Blender cannot find the Optix related parts of the driver. >> This package still does not work for me. As far as I can see, >> the non-Optix specific things are fine, though. >> >> So I will reboot, remove the portage package of the driver and >> install the original by nvidia and this will fix the issue >> with Optix... >> >> Cheers! >> Meino > > Thanks for the info. Now I think I'm good to go. > > Dale > > :-) :-) Hi all, Picking random message to reply with update. I updated to nvidia-drivers-440.82 and got it to work. Perhaps this is just a minor hickup but when I loaded the nvidia modules, it didn't load the drm and modeset ones it depends on. Maybe it was something I missed or once I get into my new kernel or reboot, it would work correctly then. I don't recall it ever doing that before. I'll make a note of that. For anyone else who upgrades without a reboot and X doesn't come up, make sure all the video modules are loaded. X does not come up without them all. Sort of freaked me out a bit for a second. lol I'm still on my old kernel but nvidia did build and install for both my current kernel and the new one which is gentoo-sources-5.6.7. I plan to boot into it when my batteries come in and I put those in my UPS. Once all that gets done, new video drivers, new kernel, new UPS batteries and since I update tonight and the batteries come in tomorrow, a up to date OS as well. Between that and the awesome new disc I'm using to prepare my garden, I can't complain. :-D Thanks to all. Dale :-) :-)
Re: [gentoo-user] which linux RAID setup to choose?
Caveman Al Toraboran wrote: > On Sunday, May 3, 2020 1:23 PM, Wols Lists wrote: > >> For anything above raid 1, MAKE SURE your drives support SCT/ERC. For >> example, Seagate Barracudas are very popular desktop drives, but I guess >> maybe HALF of the emails asking for help recovering an array on the raid >> list involve them dying ... >> >> (I've got two :-( but my new system - when I get it running - has >> ironwolves instead.) > that's very scary. > > just to double check: are those help emails about > linux's software RAID? or is it about hardware > RAIDs? > > the reason i ask about software vs. hardware, is > because of this wiki article [1] which seems to > suggest that mdadm handles error recovery by > waiting for up to 30 seconds (set in > /sys/block/sd*/device/timeout) after which the > device is reset. > > am i missing something? to me it seems that [1] > seems to suggest that linux software raid has a > reliable way to handle the issue? since i guess > all disks support resetting well? > > [1] https://en.wikipedia.org/wiki/Error_recovery_control#Software_RAID > > > I'd like to add something about the PMR/SMR thing. I bought a SMR drive without knowing it. Now when I search for a hard drive, I add NAS to the search string. That seems to weed out the SMR type drives. Once I find a exact model, I google it up to confirm. So far, that little trick has worked pretty well. It may be something you want to consider using as well. NAS drives tend to be more robust it seems. Given you are using RAID, you likely want a more robust and dependable drive, if drives can be put into that category nowadays. :/ Hope that helps. Dale :-) :-)
Re: [gentoo-user] 32GB RAM and Swap
On Sat, May 02, 2020 at 10:12:08PM +0800, William Kenworthy wrote: > I am afraid this is an ".. it depends" question. > Yes, I agree. > If you work with large images or data sets, swap can be really handy. > If you are doing a little programming, web browsing, reading email you > will *probably* be ok, but why risk it? > Risk what? Having the OOM killer kill the problematic process? Depending on your usage, this might be the best. Personally I prefer that to a system that is stuck. I never had to force reboot on a system without swap, whereas with swap I had to reboot most of the times swap was used. Also it's super annoying when your system freezes because of a background process swapping (eg. an emerge world update) while your doing something else. I've been running a 8GB system for a year, before that a 4GB system. Both without swap. It's been fine so far. I did have processes killed (eg. firefox compiling), but at least I can continue to use my system without being interrupted by a freeze. It's a lot less frustrating to have to resume a killed compilation than to deal with a frozen system. > I have a 32gb ram in a master server for an mfs filesystem - it normally > sits at about 5GB of ram - however it can go well over 32Gb into swap at > times - the first machine I tried it with only had 4gb ram and crashed > when it filled the ram, and 8g swap taking the test file system with it > - its now production so I am not going to risk it by underprovisioning > swap. My 32Gb desktop is not using any swap at the moment ... but it > has used it at times. > > So, yes its quite likely you wont use swap - but if you do something > that needs it, it can help avoid a very messy crash. > > Swap is slow, but if you actually need it - its probably critical that > you have it! Unless you are really short of disk space, treat it as > insurance :) > > Look into using swapfiles instead of partitions for flexibility, and the > sysctl values of "vm.swappiness" and "vm.vfs_cache_pressure" to manage > swap usage (you can set to not use swap until it really has to - some > have seen the kernel being too eager to swap out causing slowdowns, > though you can make it go in the other direction and "thrash" when it > actually needs to use swap if you go to far. The default kernel swap > mechanism isn't really that bad! > Swapfiles are great, because you can only add them when you need and remove them when you're done. I sometimes use them when emerging large stuff when I have other big processes in memory. > So yes, most of my machines don't need swap *right now* and swap looks > like its not being used so it could be removed, but I cant guarantee > that they never will, and having years of experience using swap I > recommend that its better to be cautious and survive :) My systems usually survive (are not forcibly rebooted) better when there is no swap. But I agree that in the end, it depends on the usage.
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 10:56 AM Caveman Al Toraboran < toraboracave...@protonmail.com> wrote: > > On Sunday, May 3, 2020 1:23 PM, Wols Lists wrote: > > > For anything above raid 1, MAKE SURE your drives support SCT/ERC. For > > example, Seagate Barracudas are very popular desktop drives, but I guess > > maybe HALF of the emails asking for help recovering an array on the raid > > list involve them dying ... > > > > (I've got two :-( but my new system - when I get it running - has > > ironwolves instead.) > > that's very scary. > > just to double check: are those help emails about > linux's software RAID? or is it about hardware > RAIDs? > > the reason i ask about software vs. hardware, is > because of this wiki article [1] which seems to > suggest that mdadm handles error recovery by > waiting for up to 30 seconds (set in > /sys/block/sd*/device/timeout) after which the > device is reset. > > am i missing something? to me it seems that [1] > seems to suggest that linux software raid has a > reliable way to handle the issue? since i guess > all disks support resetting well? > > [1] https://en.wikipedia.org/wiki/Error_recovery_control#Software_RAID > When doing Linux RAID, hardware or software, make sure you get a RAID aware drive that supports TLER (Time Limited Error Recovery) or whatever the vendor that makes your drive calls it. Typically this is set at about 7 seconds guaranteeing that no mater what's going on the drive will respond to the upper layers (mdadm) to let it know it's alive. A non-RAID drive with no TLER feature will respond when it's ready and typically if that's longer than 30 seconds then the RAID subsystem kicks the drive and you have to re-add it. While there's nothing 'technically' wrong with the storage when the RAID rebuilds you eventually hit another on of these >30 second waits and another drive gets kicked and you're dead. I've used the WD Reds and WD Golds (no not sold) and never had any problem. Build a RAID with a WD Green and you're in for trouble. ;-))) HTH, Mark
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 1:44 AM Caveman Al Toraboran wrote: > > * RAID 1: fails to satisfy points (1) and (3)... > this leaves me with RAID 10 Two things: 1. RAID 10 doesn't satisfy point 1 (read and write performance are identical). No RAID implementation I'm aware of does. 2. Some RAID1 implementations can satisfy point 3 (expandability to additional space and replication multiplicities), particular when combined with LVM. I'd stop and think about your requirements a bit. You seem really concerned about having identical read and write performance. RAID implementations all have their pros in cons both in comparison with each other, in comparison with non-RAID, and in comparison between read and write within any particular RAID implementation. I don't think you should focus so much on whether read=write in your RAID. I'd focus more on whether read and write both meet your requirements. And on that note, what are your requirements? You haven't mentioned what you plan to store on it or how this data will be stored or accessed. It is hard to say whether any design will meet your performance requirements when you haven't provided any, other than a fairly arbitrary read=write one. In general most RAID1 implementations aren't going to lag regular non-RAID disk by much and will often exceed it (especially for reading). I'm not saying RAID1 is the best option for you - I'm just suggesting that you don't toss it out just because it reads faster than it writes, especially in favor of RAID 10 which also reads faster than it writes but has the additional caveat that small writes may necessitate an additional read before write. Not knowing your requirements it is hard to make more specific recommendations but I'd also consider ZFS and distributed filesystems. They have some pros and cons around flexibility and if you're operating at a small scale - it might not be appropriate for your use case, but you should consider them. -- Rich
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 2:29 PM Mark Knecht wrote: > > I've used the WD Reds and WD Golds (no not sold) and never had any problem. > Up until a few weeks ago I would have advised the same, but WD was just caught shipping unadvertised SMR in WD Red disks. This is going to at the very least impact your performance if you do a lot of writes, and it can be incompatible with rebuilds in particular with some RAID implementations. Seagate and Toshiba have also been quietly using it but not in their NAS-labeled drives and not as extensively in general. At the very least you should check the model number lists that have been recently released to check if the drive you want to get uses SMR. I'd also get it from someplace with a generous return policy and do some benchmarking to confirm that the drive isn't SMR (you're probably going to have to do continuous random writes exceeding the total capacity of the drive before you see problems - or at least quite a bit of random writing - the amount of writing needed will be less once the drive has been in use for a while but a fresh drive basically acts like close to a full-disk-sized write cache as far as SMR goes). > Build a RAID with a WD Green and you're in for trouble. ;-))) It really depends on your RAID implementation. Certainly I agree that it is better to have TLER, but for some RAID implementations not having it just causes performance drops when you actually have errors (which should be very rare). For others it can cause drives to be dropped. I wouldn't hesitate to use greens in an mdadm or zfs array with default options, but with something like hardware RAID I'd be more careful. If you use aggressive timeouts on your RAID then the Green is more likely to get kicked out. I agree with the general sentiment to have a spare if it will take you a long time to replace failed drives. Alternatively you can have additional redundancy, or use a RAID alternative that basically treats all free space as an effective spare (like many distributed filesystems). -- Rich
Re: [gentoo-user] which linux RAID setup to choose?
On 03/05/2020 18:55, Caveman Al Toraboran wrote: On Sunday, May 3, 2020 1:23 PM, Wols Lists wrote: For anything above raid 1, MAKE SURE your drives support SCT/ERC. For example, Seagate Barracudas are very popular desktop drives, but I guess maybe HALF of the emails asking for help recovering an array on the raid list involve them dying ... (I've got two :-( but my new system - when I get it running - has ironwolves instead.) that's very scary. just to double check: are those help emails about linux's software RAID? or is it about hardware RAIDs? They are about linux software raid. Hardware raid won't be any better. the reason i ask about software vs. hardware, is because of this wiki article [1] which seems to suggest that mdadm handles error recovery by waiting for up to 30 seconds (set in /sys/block/sd*/device/timeout) after which the device is reset. Which if your drive does not support SCT/ERC then goes *badly* wrong. am i missing something? Yes ... to me it seems that [1] seems to suggest that linux software raid has a reliable way to handle the issue? Well, if the paragraph below were true, it would. since i guess all disks support resetting well? That's the point. THEY DON'T! That's why you need SCT/ERC ... [1] https://en.wikipedia.org/wiki/Error_recovery_control#Software_RAID https://raid.wiki.kernel.org/index.php/Choosing_your_hardware,_and_what_is_a_device%3F#Desktop_and_Enterprise_drives https://raid.wiki.kernel.org/index.php/Timeout_Mismatch Cheers, Wol
Re: [gentoo-user] which linux RAID setup to choose?
On 03/05/2020 21:07, Rich Freeman wrote: I don't think you should focus so much on whether read=write in your RAID. I'd focus more on whether read and write both meet your requirements. If you think about it, it's obvious that raid-1 will read faster than it writes - it has to write two copies while it only reads one. Likewise, raids 5 and 6 will be slower writing than reading - for a normal read it only reads the data disks, but when writing it has to write (and calculate!) parity as well. A raid 1 should read data faster than a lone disk. A raid 5 or 6 should read noticeably faster because it's reading across more than one disk. If you're worried about write speeds, add a cache. Cheers, Wol
Re: [gentoo-user] which linux RAID setup to choose?
On Sunday, May 3, 2020 1:23 PM, Wols Lists wrote: > For anything above raid 1, MAKE SURE your drives support SCT/ERC. For > example, Seagate Barracudas are very popular desktop drives, but I guess > maybe HALF of the emails asking for help recovering an array on the raid > list involve them dying ... > > (I've got two :-( but my new system - when I get it running - has > ironwolves instead.) that's very scary. just to double check: are those help emails about linux's software RAID? or is it about hardware RAIDs? the reason i ask about software vs. hardware, is because of this wiki article [1] which seems to suggest that mdadm handles error recovery by waiting for up to 30 seconds (set in /sys/block/sd*/device/timeout) after which the device is reset. am i missing something? to me it seems that [1] seems to suggest that linux software raid has a reliable way to handle the issue? since i guess all disks support resetting well? [1] https://en.wikipedia.org/wiki/Error_recovery_control#Software_RAID
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 5:32 PM antlists wrote: > > On 03/05/2020 21:07, Rich Freeman wrote: > > I don't think you should focus so much on whether read=write in your > > RAID. I'd focus more on whether read and write both meet your > > requirements. > > If you think about it, it's obvious that raid-1 will read faster than it > writes - it has to write two copies while it only reads one. Yes. The same is true for RAID10, since it has to also write two copies of everything. > > Likewise, raids 5 and 6 will be slower writing than reading - for a > normal read it only reads the data disks, but when writing it has to > write (and calculate!) parity as well. Yes, but with any of the striped modes (0, 5, 6, 10) there is an additional issue. Writes have to generally be made in entire stripes, so if you overwrite data in-place in units smaller than an entire stripe, then the entire stripe needs to first be read, and then it can be overwritten again. This is an absolute requirement if there is parity involved. If there is no parity (RAID 0,10) then an implementation might be able to overwrite part of a stripe in place without harming the rest. > > A raid 1 should read data faster than a lone disk. A raid 5 or 6 should > read noticeably faster because it's reading across more than one disk. More-or-less. RAID 1 is going to generally benefit from lower latency because reads can be divided across mirrored copies (and there could be more than one replica). Any of the striped modes are going to be the same as a single disk on latency, but will have much greater bandwidth. That bandwidth gain applies to both reading and writing, as long as the data is sequential. This is why it is important to understand your application. There is no one "best" RAID implementation. They all have pros and cons depending on whether you care more about latency vs bandwidth and also read vs write. And of course RAID isn't the only solution out there for this stuff. Distributed filesystems also have pros and cons, and often those have multiple modes of operation on top of this (usually somewhat mirroring the options available for RAID but across multiple hosts). For general storage I'm using zfs with raid1 pairs of disks (the pool can have multiple pairs), and for my NAS for larger-scale media/etc storage I'm using lizardfs. I'd use ceph instead in any kind of enterprise setup, but that is much more RAM-hungry and I'm cheap. -- Rich
Re: [gentoo-user] which linux RAID setup to choose?
Am 03.05.2020 um 23:46 schrieb Caveman Al Toraboran: so, in summary: /\ | a 5-disk RAID10 is better than a 6-disk RAID10 | | ONLY IF your data is WORTH LESS than 3,524.3 | | bucks. | \/ any thoughts? i'm a newbie. i wonder how industry people think? Don't forget that having more drives increases the odds of a failing drive. If you have infinite drives at any given moment infinite drives will fail. Anyway I wouldn't know how to calculate this. Most people are limited by money and space. Even if this isn't your problem you will always need an additional backup strategy. The hole system can fail. I run a system with 8 drives where two can fail and they can be hot swoped. This is a closed source SAS which I really like except the part being closed source. I don't even know what kind of raid is used. The only person I know who is running a really huge raid ( I guess 2000+ drives) is comfortable with some spare drives. His raid did fail an can fail. Data will be lost. Everything important has to be stored at a secondary location. But they are using the raid to store data for some days or weeks when a server is calculating stuff. If the raid fails they have to restart the program for the calculation. Facebook used to store data which is sometimes accessed on raids. Since they use energy they stored data which is nearly never accessed on blue ray disks. I don't know if they still do. Reading is very slow if a mechanical arm first needs to fetch a specific blue ray out of hundreds and put in a disk reader but it is very energy efficient.
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 1:16 PM Rich Freeman wrote: > > On Sun, May 3, 2020 at 2:29 PM Mark Knecht wrote: > > > > I've used the WD Reds and WD Golds (no not sold) and never had any problem. > > > > Up until a few weeks ago I would have advised the same, but WD was > just caught shipping unadvertised SMR in WD Red disks. This is going > to at the very least impact your performance if you do a lot of > writes, and it can be incompatible with rebuilds in particular with > some RAID implementations. Seagate and Toshiba have also been quietly > using it but not in their NAS-labeled drives and not as extensively in > general. I read somewhere that they knew they'd been caught and were coming clean. As I'm not buying anything at this time I didn't pay too much attention. This link is at least similar to what I read earlier. Possibly it's of interest. https://www.extremetech.com/computing/309730-western-digital-comes-clean-shares-which-hard-drives-use-smr Another case of unbridled capitalism and consumers being hurt. Cheers, Mark
Re: [gentoo-user] which linux RAID setup to choose?
On 03/05/2020 22:46, Caveman Al Toraboran wrote: On Sunday, May 3, 2020 6:27 PM, Jack wrote: curious. how do people look at --layout=n2 in the storage industry? e.g. do they ignore the optimistic case where 2 disk failures can be recovered, and only assume that it protects for 1 disk failure? You CANNOT afford to be optimistic ... Murphy's law says you will lose the wrong second disk. i see why gambling is not worth it here, but at the same time, i see no reason to ignore reality (that a 2 disk failure can be saved). Don't ignore that some 2-disk failures CAN'T be saved ... e.g. a 4-disk RAID10 with -layout=n2 gives 1*4/10 + 2*4/10 = 1.2 expected recoverable disk failures. details are below: now, if we do a 5-disk --layout=n2, we get: 1(1)2(2)3 (3)4(4)5(5) 6(6)7(7)8 (8)9(9)10 (10) 11 (11) 12 (12) 13 (13) ... obviously, there are 5 possible ways a single disk may fail, out of which all of the 5 will be recovered. Don't forget a 4+spare layout, which *should* survive a 2-disk failure. there are nchoosek(5,2) = 10 possible ways a 2 disk failure could happen, out of which 5 will be recovered: so, by transforming a 4-disk RAID10 into a 5-disk one, we increase total storage capacity by a 0.5 disk's worth of storage, while losing the ability to recover 0.2 disks. but if we extended the 4-disk RAID10 into a 6-disk --layout=n2, we will have: 6 nchoosek(6,2) - 3 = 1 * - + 2 * - 6 + nchoosek(6,2) 6 + nchoosek(6,2) = 6/21 + 2 * 12/15 = 1.8857 expected recoverable failing disks. almost 2. i.e. there is 80% chance of surviving a 2 disk failure. so, i wonder, is it a bad decision to go with an even number disks with a RAID10? what is the right way to think to find an answer to this question? i guess the ultimate answer needs knowledge of these: * F1: probability of having 1 disks fail within the repair window. * F2: probability of having 2 disks fail within the repair window. * F3: probability of having 3 disks fail within . the repair window. . . * Fn: probability of having n disks fail within the repair window. * R1: probability of surviving 1 disks failure. equals 1 with all related cases. * R2: probability of surviving 2 disks failure. equals 1/3 with 5-disk RAID10 equals 0.8 with a 6-disk RAID10. * R3: probability of surviving 3 disks failure. equals 0 with all related cases. . . . * Rn: probability of surviving n disks failure. equals 0 with all related cases. * L : expected cost of losing data on an array. * D : price of a disk. Don't forget, if you have a spare disk, the repair window is the length of time it takes to fail-over ... this way, the absolute expected cost when adopting a 6-disk RAID10 is: = 6D + F1*(1-R1)*L + F2*(1-R2)*L + F3*(1-R3)*L + ... = 6D + F1*(1-1)*L + F2*(1-0.8)*L + F3*(1-0)*L + ... = 6D + 0 + F2*(0.2)*L + F3*(1-0)*L + ... and the absolute cost for a 5-disk RAID10 is: = 5D + F1*(1-1)*L + F2*(1-0.)*L + F3*(1-0)*L + ... = 5D + 0 + F2*(0.6667)*L + F3*(1-0)*L + ... canceling identical terms, the difference cost is: 6-disk ===> 6D + 0.2*F2*L 5-disk ===> 5D + 0.6667*F2*L from here [1] we know that a 1TB disk costs $35.85, so: 6-disk ===> 6*35.85 + 0.2*F2*L 5-disk ===> 5*35.85 + 0.6667*F2*L now, at which point is a 5-disk array a better economical decision than a 6-disk one? for simplicity, let LOL = F2*L: 5*35.85 + 0.6667 * LOL < 6*35.85 + 0.2 * LOL 0.6667*LOL - 0.2 * LOL < 6*35.85 - 5*35.85 LOL * (0.6667 - 0.2)< 6*35.85 - 5*35.85 6*35.85 - 5*35.85 LOL < - 0.6667 - 0.2 LOL < 76.816 F2*L < 76.816 so, a 5-disk RAID10 is better than a 6-disk RAID10 only if: F2*L < 76.816 bucks. this site [2] says that 76% of seagate disks fail per year (:D). and since disks fail independent of each other mostly, then, the probabilty of having 2 disks fail in a year is: 76% seems incredibly high. And no, disks do not fail independently of each other. If you buy a bunch of identical disks, at the same time, and stick them all in the same raid array, the chances of them all wearing out at the same time are rather higher than random chance would suggest. Which is why, if a raid disk fails, the advice is always to replace it asap. And if possible, to recover the failed drive to try and copy that rather than hammer the rest of the raid. Bear in mind that, it doesn't matter how many drives a raid-10 has, if you're recovering on to a new drive, the data is stored on just two of the o
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 6:52 PM Mark Knecht wrote: > > On Sun, May 3, 2020 at 1:16 PM Rich Freeman wrote: > > > > Up until a few weeks ago I would have advised the same, but WD was > > just caught shipping unadvertised SMR in WD Red disks. This is going > > to at the very least impact your performance if you do a lot of > > writes, and it can be incompatible with rebuilds in particular with > > some RAID implementations. Seagate and Toshiba have also been quietly > > using it but not in their NAS-labeled drives and not as extensively in > > general. > > I read somewhere that they knew they'd been caught and were coming clean. Yup. WD was caught. Then they first came out with a "you're using it wrong" sort of defense but they did list the SMR drives. Then they came out with a bit more of an even-handed response. The others weren't caught as far as I'm aware but probably figured the writing was on the wall since no doubt everybody and their uncle is going to be benchmarking every drive they own. > Another case of unbridled capitalism and consumers being hurt. I agree. This video has a slightly different perspective. It doesn't disagree on that conclusion, but it does explain more of the industry thinking that got us here (beyond the simple/obvious it saves money): https://www.youtube.com/watch?v=gSionmmunMs -- Rich
[gentoo-user] AMD Radeon R7 370 (Pitcairn) causing the bootup to hang
Hi gentoo-user, I'm attempting to configure a mid-range video card: the Radeon R7 370. Running on the Pitcairn chipset and a member of the Southern Islands family, I am surprised at the complexity of setting up the Radeon driver in comparison to its NVIDIA counterpart. I followed [1] carefully. Initially opting to compile AMDGPU into the kernel, I emerged linux-firmware with the following files. All of the relevant files were added to the kernel's CONFIG_EXTRA_FIRMWARE string, using /lib/firmware as the base directory. radeon/pitcairn_smc.bin radeon/pitcairn_ce.bin radeon/pitcairn_mc.bin radeon/pitcairn_me.bin radeon/pitcairn_pfp.bin radeon/pitcairn_k_smc.bin radeon/pitcairn_rlc.bin radeon/TAHITI_uvd.bin radeon/TAHITI_vce.bin Unfortunately, upon booting, the kernel hangs with the following message. This seems to be rather common, with a similar complaint being discussed at [2]. fb0: switching to amdgpudrmfb from EFI VGA As this all occurs pre-OpenRC, I am incapable of creating an S.S.H.\ connection to the machine from my laptop. When booting the kernel with the `nomodesetting` parameter, the X server reports the following after a successful kernel boot (created when executing `startx`): [timestamps omitted] (EE) Failed to load module "fbdev" (module does not exist, 0) (II) LoadModule: "vesa" (WW) Warning, couldn't open module vesa (EE) Failed to load module "vesa" (module does not exist, 0) (II) RADEON: Driver for ATI/AMD Radeon chipsets: ATI Radeon Mobility X600 (M24), ATI FireMV 2400, ATI Radeon Mobility X300 (M24), ATI FireGL M24 GL, [...] ARUBA, TAHITI, PITCAIRN, VERDE, OLAND, HAINAN, BONAIRE, KABINI, MULLINS, KAVERI, HAWAII (II) modesetting: Driver for Modesetting Kernel Drivers: kms (--) using VT number 7 (II) [KMS] drm report modesetting isn't supported. (EE) open /dev/dri/card0: No such file or directory (WW) Falling back to old probe method for modesetting (EE) open /dev/dri/card0: No such file or directory (EE) Screen 0 deleted because of no matching config section. (II) UnloadModule: "radeon" (EE) Screen 0 deleted because of no matching config section. (II) UnloadModule: "modesetting" (EE) Device(s) detected, but none match those in the config file. (EE) Fatal server error: (EE) no screens found(EE) (EE) In a fruitless attempt to fix this, I opted to instead load AMDGPU as a kernel module. That did not go far either, as apparently a bug exists in the Radeon drivers which causes OpenRC to hang when setting the system time according to the internal clock [3]: Setting system clock using hardware clock [Local Time] [hangs forever] Any help with this would be appreciated. I'm moving away from NVIDIA due to the requirement of proprietary drivers to get any decent performance, however now it feels as though the AMD drivers, although open-source, consist of too many bugs (such as hanging the boot-up process for some reason or another) to be of any actual use. Whilst I'm aware that is obviously not the case due to the popularity of their cards, I am bewildered at how difficult this seems. Cheers. [1] https://wiki.gentoo.org/wiki/AMDGPU [2] https://forums.gentoo.org/viewtopic-t-1061750-start-0.html [3] https://www.reddit.com/r/Gentoo/comments/9oax20/ -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] which linux RAID setup to choose?
On Sun, May 3, 2020 at 6:50 PM hitachi303 wrote: > > The only person I know who is running a really huge raid ( I guess 2000+ > drives) is comfortable with some spare drives. His raid did fail an can > fail. Data will be lost. Everything important has to be stored at a > secondary location. But they are using the raid to store data for some > days or weeks when a server is calculating stuff. If the raid fails they > have to restart the program for the calculation. So, if you have thousands of drives, you really shouldn't be using a conventional RAID solution. Now, if you're just using RAID to refer to any technology that stores data redundantly that is one thing. However, if you wanted to stick 2000 drives into a single host using something like mdadm/zfs, or heaven forbid a bazillion LSI HBAs with some kind of hacked-up solution for PCIe port replication plus SATA bus multipliers/etc, you're probably doing it wrong. (Really even with mdadm/zfs you probably still need some kind of terribly non-optimal solution for attaching all those drives to a single host.) At that scale you really should be using a distributed filesystem. Or you could use some application-level solution that accomplishes the same thing on top of a bunch of more modest hosts running zfs/etc (the Backblaze solution at least in the past). The most mainstream FOSS solution at this scale is Ceph. It achieves redundancy at the host level. That is, if you have it set up to tolerate two failures then you can take two random hosts in the cluster and smash their motherboards with a hammer in the middle of operation, and the cluster will keep on working and quickly restore its redundancy. Each host can have multiple drives, and losing any or all of the drives within a single host counts as a single failure. You can even do clever stuff like tell it which hosts are attached to which circuit breakers and then you could lose all the hosts on a single power circuit at once and it would be fine. This also has the benefit of covering you when one of your flakey drives causes weird bus issues that affect other drives, or one host crashes, and so on. The redundancy is entirely at the host level so you're protected against a much larger number of failure modes. This sort of solution also performs much faster as data requests are not CPU/NIC/HBA limited for any particular host. The software is obviously more complex, but the hardware can be simpler since if you want to expand storage you just buy more servers and plug them into the LAN, versus trying to figure out how to cram an extra dozen hard drives into a single host with all kinds of port multiplier games. You can also do maintenance and just reboot an entire host while the cluster stays online as long as you aren't messing with them all at once. I've gone in this general direction because I was tired of having to try to deal with massive cases, being limited to motherboards with 6 SATA ports, adding LSI HBAs that require an 8x slot and often conflicts with using an NVMe, and so on. -- Rich
Re: [gentoo-user] Ventoy: bootable USB drive for ISO files
james wrote: > I wonder if any Gentoo dev is working on implementing this? > > Has anyone tested it? > > https://www.ventoy.net/en/index.html > > I just read about it on ycombinator > > > curiously, > James > > I went here: https://www.ventoy.net/en/isolist.html If you scroll down the list, about half way down the page, it shows Gentoo as a option. It has a minimal install and a live DVD option. It supports legacy and UEFI thingy. I'm not sure how it works yet tho. I need to read a bit more. Dale :-) :-)
Re: [gentoo-user] Ventoy: bootable USB drive for ISO files
On 5/3/20 8:47 PM, Dale wrote: james wrote: I wonder if any Gentoo dev is working on implementing this? Has anyone tested it? https://www.ventoy.net/en/index.html I just read about it on ycombinator curiously, James I went here: https://www.ventoy.net/en/isolist.html If you scroll down the list, about half way down the page, it shows Gentoo as a option.� It has a minimal install and a live DVD option.� It supports legacy and UEFI thingy.� I'm not sure how it works yet tho.� I need to read a bit more. Dale :-)� :-) Yep, but I cannot find either file anywhere? Still searching for those (2017)an jan2020 images: Gentoo install-amd64-minimal-20200119T214502Z.iso livedvd-amd64-hardened-nomultilib-20170118.iso Let me know/post if there is a place actually download these iso files?
Re: [gentoo-user] Ventoy: bootable USB drive for ISO files
On Sun, May 03, 2020 at 09:04:27PM -0400, james wrote: > Gentoo > install-amd64-minimal-20200119T214502Z.iso > livedvd-amd64-hardened-nomultilib-20170118.iso > > Let me know/post if there is a place actually download these iso files? It seems like a LiveDVD image hasn't been built in a while: https://wiki.gentoo.org/wiki/Project:RelEng/LiveDVD -- Ashley Dixon suugaku.co.uk 2A9A 4117 DA96 D18A 8A7B B0D2 A30E BF25 F290 A8AA signature.asc Description: PGP signature
Re: [gentoo-user] Ventoy: bootable USB drive for ISO files
james wrote: > On 5/3/20 8:47 PM, Dale wrote: >> james wrote: >>> I wonder if any Gentoo dev is working on implementing this? >>> >>> Has anyone tested it? >>> >>> https://www.ventoy.net/en/index.html >>> >>> I just read about it on ycombinator >>> >>> >>> curiously, >>> James >>> >>> >> >> >> I went here: >> >> https://www.ventoy.net/en/isolist.html >> >> If you scroll down the list, about half way down the page, it shows >> Gentoo as a option.� It has a minimal install and a live DVD >> option.� It >> supports legacy and UEFI thingy.� I'm not sure how it works yet >> tho.� I >> need to read a bit more. >> >> Dale >> >> :-)� :-) >> > > > Yep, but I cannot find either file anywhere? > > Still searching for those (2017)an jan2020 images: > > > Gentoo > install-amd64-minimal-20200119T214502Z.iso > livedvd-amd64-hardened-nomultilib-20170118.iso > > Let me know/post if there is a place actually download these iso files? > I would guess you download from the distros site. Here for Gentoo. https://gentoo.org/downloads/ I'm not quite sure how it works yet tho. Doing updates and such at the moment. Dale :-) :-)
Re: [gentoo-user] which linux RAID setup to choose?
On Sunday, May 3, 2020 6:27 PM, Jack wrote: > Minor point - you have one duplicate line there ". f f ." which is the > second and last line of the second group. No effect on anything else in > the discussion. thanks. > Trying to help thinking about odd numbers of disks, if you are still > allowing only one disk to fail, then you can think about mirroring half > disks, so each disk has half of it mirrored to a different disk, instead > of drives always being mirrored in pairs. that definitely helped get me unstuck and continue thinking. thanks. curious. how do people look at --layout=n2 in the storage industry? e.g. do they ignore the optimistic case where 2 disk failures can be recovered, and only assume that it protects for 1 disk failure? i see why gambling is not worth it here, but at the same time, i see no reason to ignore reality (that a 2 disk failure can be saved). e.g. a 4-disk RAID10 with -layout=n2 gives 1*4/10 + 2*4/10 = 1.2 expected recoverable disk failures. details are below: F . . . < recoverable . F . . < cases with . . F . < 1 disk . . . F < failure F . . F < recoverable . F F . < cases with . F . F < 2 disk F . F . < failures F F . . < not recoverable . . F F < cases with 2 disk < failures now, if we do a 5-disk --layout=n2, we get: 1(1)2(2)3 (3)4(4)5(5) 6(6)7(7)8 (8)9(9)10 (10) 11 (11) 12 (12) 13 (13) ... obviously, there are 5 possible ways a single disk may fail, out of which all of the 5 will be recovered. there are nchoosek(5,2) = 10 possible ways a 2 disk failure could happen, out of which 5 will be recovered: xxx (1) xxx (2)3 xxx4xxx5(5) xxx (1)2xxx3 xxx4(4) xxx (5) 1xxx2xxx3 (3) xxx (4) xxx (5) 1xxx2(2) xxx (3) xxx (4)5xxx 1(1) xxx (2) xxx (3)4xxx5xxx so, expected recoverable disk failures for a 5-disk RAID10 --layout=n2 is: 1*5/15 + 2*5/15 = 1 so, by transforming a 4-disk RAID10 into a 5-disk one, we increase total storage capacity by a 0.5 disk's worth of storage, while losing the ability to recover 0.2 disks. but if we extended the 4-disk RAID10 into a 6-disk --layout=n2, we will have: 6 nchoosek(6,2) - 3 = 1 * - + 2 * - 6 + nchoosek(6,2) 6 + nchoosek(6,2) = 6/21 + 2 * 12/15 = 1.8857 expected recoverable failing disks. almost 2. i.e. there is 80% chance of surviving a 2 disk failure. so, i wonder, is it a bad decision to go with an even number disks with a RAID10? what is the right way to think to find an answer to this question? i guess the ultimate answer needs knowledge of these: * F1: probability of having 1 disks fail within the repair window. * F2: probability of having 2 disks fail within the repair window. * F3: probability of having 3 disks fail within . the repair window. . . * Fn: probability of having n disks fail within the repair window. * R1: probability of surviving 1 disks failure. equals 1 with all related cases. * R2: probability of surviving 2 disks failure. equals 1/3 with 5-disk RAID10 equals 0.8 with a 6-disk RAID10. * R3: probability of surviving 3 disks failure. equals 0 with all related cases. . . . * Rn: probability of surviving n disks failure. equals 0 with all related cases. * L : expected cost of losing data on an array. * D : price of a disk. this way, the absolute expected cost when adopting a 6-disk RAID10 is: = 6D + F1*(1-R1)*L + F2*(1-R2)*L + F3*(1-R3)*L + ... = 6D + F1*(1-1)*L + F2*(1-0.8)*L + F3*(1-0)*L + ... = 6D + 0 + F2*(0.2)*L + F3*(1-0)*L + ... and the absolute cost for a 5-disk RAID10 is: = 5D + F1*(1-1)*L + F2*(1-0.)*L + F3*(1-0)*L + ... = 5D + 0 + F2*(0.6667)*L + F3*(1-0)*L + ... canceling identical terms, the difference cost is: 6-disk ===> 6D + 0.2*F2*L 5-disk ===> 5D + 0.6667*F2*L from here [1] we know that a 1TB disk costs $35.85, so: 6-disk ===> 6*35.85 + 0.2*F2*L 5-disk ===> 5*35.85 + 0.6667*F2*L now, at which point is a 5-disk array a better economical decision than a 6-disk one? for simplicity, let LOL = F2*L: 5*35.85 + 0.6667 * LOL < 6*35.85 + 0.2 * LOL 0.6667*LOL - 0.2 * LOL < 6*35.85 - 5*35.85 LOL * (0.6667 - 0.2)< 6*35.85 - 5*35.85 6*35.85 - 5*35.85 LOL < - 0.6667 - 0.2 LOL < 76.816 F
Re: [gentoo-user] which linux RAID setup to choose?
On Monday, May 4, 2020 2:50 AM, hitachi303 wrote: > Am 03.05.2020 um 23:46 schrieb Caveman Al Toraboran: > > > so, in summary: > > /\ > > | a 5-disk RAID10 is better than a 6-disk RAID10 | > > | ONLY IF your data is WORTH LESS than 3,524.3 | > > | bucks. | > > \/ > > any thoughts? i'm a newbie. i wonder how > > industry people think? > > Don't forget that having more drives increases the odds of a failing > drive. If you have infinite drives at any given moment infinite drives > will fail. Anyway I wouldn't know how to calculate this. by drive, you mean a spinning hard disk? i'm not sure how "infinite" helps here even theoretically. e.g. say that every year, 76% of disks fail. in the limit as the number of disks approaches infinity, then 76% of infinity is infinity. but, how is this useful? > Most people are limited by money and space. Even if this isn't your > problem you will always need an additional backup strategy. The hole > system can fail. > I run a system with 8 drives where two can fail and they can be hot > swoped. This is a closed source SAS which I really like except the part > being closed source. I don't even know what kind of raid is used. > > The only person I know who is running a really huge raid ( I guess 2000+ > drives) is comfortable with some spare drives. His raid did fail an can > fail. Data will be lost. Everything important has to be stored at a > secondary location. But they are using the raid to store data for some > days or weeks when a server is calculating stuff. If the raid fails they > have to restart the program for the calculation. thanks a lot. highly appreciate these tips about how others run their storage. however, i am not sure what is the takeaway from this. e.g. your closed-source NAS vs. a large RAID. they don't seem to be mutually exclusive to me (both might be on RAID). to me, a NAS is just a computer with RAID. no? > Facebook used to store data which is sometimes accessed on raids. Since > they use energy they stored data which is nearly never accessed on blue > ray disks. I don't know if they still do. Reading is very slow if a > mechanical arm first needs to fetch a specific blue ray out of hundreds > and put in a disk reader but it is very energy efficient. interesting.
[gentoo-user] Ventoy: bootable USB drive for ISO files
I wonder if any Gentoo dev is working on implementing this? Has anyone tested it? https://www.ventoy.net/en/index.html I just read about it on ycombinator curiously, James
Re: [gentoo-user] which linux RAID setup to choose?
On Monday, May 4, 2020 3:19 AM, antlists wrote: > On 03/05/2020 22:46, Caveman Al Toraboran wrote: > > > On Sunday, May 3, 2020 6:27 PM, Jack ostrof...@users.sourceforge.net wrote: > > curious. how do people look at --layout=n2 in the > > storage industry? e.g. do they ignore the > > optimistic case where 2 disk failures can be > > recovered, and only assume that it protects for 1 > > disk failure? > > You CANNOT afford to be optimistic ... Murphy's law says you will lose > the wrong second disk. so i guess your answer is: "yes, the industry ignores the existence of optimistic cases". if that's true, then the industry is wrong, must learn the following: 1. don't bet that your data's survival is lingering on luck (you agree with this i know). 2. don't ignore statistics that reveal the fact that lucky cases exist. (1) and (2) are not mutually exclusive, and murfphy's law would suggest to not ignore (2). becuase, if you ignore (2), you'll end up adopting a 5-disk RAID10 instead of the superior 6-disk RAID10 and end up being less lucky in practice. don't rely on lucks, but why deny good luck to come to you when it might? --- two different things. > > i see why gambling is not worth it here, but at > > the same time, i see no reason to ignore reality > > (that a 2 disk failure can be saved). > > Don't ignore that some 2-disk failures CAN'T be saved ... yeah, i'm not. i'm just not ignoring that 2-disk failure might get saved. you know... it's better to have a lil window where some good luck may chime in than banning good luck. > Don't forget, if you have a spare disk, the repair window is the length > of time it takes to fail-over ... yup. just trying to not rely on good luck that a spare is available. e.g. considering for the case that no space is there. > > this site [2] says that 76% of seagate disks fail > > per year (:D). and since disks fail independent > > of each other mostly, then, the probabilty of > > having 2 disks fail in a year is: > > 76% seems incredibly high. And no, disks do not fail independently of > each other. If you buy a bunch of identical disks, at the same time, and > stick them all in the same raid array, the chances of them all wearing > out at the same time are rather higher than random chance would suggest. i know. i had this as a note, but then removed it. anyway, some nitpics: 1. dependence != correlation. you mean correlation, not dependence. disk failure is correlated if they are baught together, but other disks don't cause the failure (unless from things like heat from other disks, or repair stress because of other disk failing). 2. i followed the extreme case where a person got his disks purchased at a random time, so that he was maximally lucky in that his disks didn't synchronize. why? (i) offers a better pessimistic result. now we know that this probability is actually lower than reality, which means that we know that the 3.5k bucks is actually even lower. this should scare us more (hence us relying on less luck). (ii) makes calculation easier.