On 2014-08-19 12:21, M G Berberich wrote: > Hello, > > we are thinking about using BtrFS on standard hardware for a > fileserver with about 50T (100T raw) of storage (25×4TByte). > > This is what I understood so far. Is this right? > > · incremental send/receive works. > > · There is no support for hotspares (spare disks that automatically > replaces faulty disk). > > · BtrFS with RAID1 is fairly stable. > > · RAID 5/6 spreads all data over all devices, leading to performance > problems on large diskarrays, and there is no option to limit the > numbers of disk per stripe so far. > > Some questions: > > · There where reports, that bcache with btrfs leads to corruption. Is > this still so? Based on some testing I did last month, bcache with anything has the potential to cause data corruption. > > · If a disk failes, does BtrFS rebalance automatically? (This would > give a a kind o hotspare behavior) No, but it wouldn't be hard to write a simple monitoring program to do this from userspace. IIRC, the big issue is that you need to add a device in-place of the failed one for the re-balance to work. > > · Besides using bcache, are there any possibilities to boost > performance by adding (dedicated) cache-SSDs to a BtrFS? Like mentioned in one of the other responses, I would suggest looking into dm-cache. BTRFS itself does not have any functionality for this, although there has been talk of implementing device priorities for reads, which could provide a similar performance boost. > > · Are there any reports/papers/web-pages about BtrFS-systems this size > in use? Praises, complains, performance-reviews, whatever… While it doesn't quite fit the description, I have had very good success with a very active 2TB BTRFS RAID10 filesystem consisting of BTRFS on four unpartitioned 1TB SATA III hard drives. The filesystem gets in excess of 100GB of data written to it each day (almost all rewrites however), and is what I use for /home, /var/log, and /var/lib, and I've had no issues with it that were caused by BTRFS, and in-fact, the very fact that it uses BTRFS helped me recover data when the storage controller they are connected to went bad. On average, I get about 125% of raw disk performance on writes, and about 110% on reads.
If you are using a very large number of disks, then I would not suggest that you use BTRFS RAID10, but instead BTRFS RAID1, as RAID10 will try to stripe things across ALL of the devices in the filesystem, and unless you have no more than about four times as many disks as storage controllers (that is, each controller has no more than four disks attached to it), the overhead outweighs the benefit of striping the data. Also, just to make sure it's clear, in BTRFS RAID1, each block gets written EXACTLY twice. On the plus side though, this means that if you do set-up a caching mechanism, you may be able to keep most of the array spun down a majority of the time.
smime.p7s
Description: S/MIME Cryptographic Signature
