UGlee posted on Sat, 01 Apr 2017 14:06:11 +0800 as excerpted: > We are working on a small NAS server for home user. The product is > equipped with a small fast SSD (around 60-120GB) and a large HDD (2T to > 4T). > > We have two choices: > > 1. using bcache to accelerate io operation 2. combining SSD and HDD into > a single btrfs volume. > > Bcache is certainly designed for our purpose. But bcache requires > complex configuration and can only start from clean disk. Also in our > test in Ubuntu 16.04, data inconsistence was observed at least once, > resulting total HDD data lost. > > So we wonder if simply putting SSD and HDD into a single btrfs volume, > in whatever mode, the general read operation (mostly readdir and > getxattr) will also be significantly faster than a single HDD without > SSD.
At present, bcache, or possibly the lvmcache alternative, are the only recommended way of creating a single btrfs out of a mixed-size ssd/hdd multi-volume. The problem is that while they've been considered, there's no present method of telling btrfs to use the smaller ssd for hotter content. The btrfs chunk allocator simply doesn't have that option at present. Which would leave you with the choice of single, raid1 or raid0 modes. Raid1 requires two copies on separate devices which would mean the extra space on the larger hdd would be wasted/unusable, and the read-mode mirror choice algorithm is purely even/odd PID-based so on single reads you'd have a 50% chance of fast ssd reads, 50% chance slow hdd. With single mode the allocator allocates to the device with the most space available first, so until the free space equalized between the two, all chunks would end up on the larger/slower hdd. And raid0 would allocate evenly (allocate-wide policy) to both, again wasting the extra space on the larger device while only giving you overall about the same speed as two hdds would give you, tho less predictably you'd get the full speed of the ssd. The default two-device setup, FWIW, is raid1-mode metadata for safety, single-mode data. As you can see, none of those are ideal from a fast-small-ssd as cache to a large-slow-hdd perspective, thus the recommendation of bcache or lvmcache if that's what you want/need. The other alternative, of course, is separate filesystems, using a combination of symlinks, partitioning and bind-mounts, to arrange for frequently accessed and performance-critical stuff such as root and /home to be on the smaller/faster ssd, while the larger/slower hdd is used for stuff like a user's multimedia partition/filesystem. That's actually what I've done here and I'm *very* happy with the result, but it's the type of solution that must either be customized per-installation, or perhaps be setup by a special-purpose distro installer designed with that sort of use-case target in mind. It's /not/ the sort of thing you can do in a NAS product and expect mass-market users to actually read and understand the docs in ordered to use the product in an optimal way. Meanwhile, since you appear to be designing a mass-market product, it's worth mentioning that btrfs is considered, certainly by its devs and users on this list, to be "still stabilizing, not fully stable and mature." As such, making and having backups at the ready for any data considered to be more valuable than the time and resources necessary to make those backups is strongly recommended, even more so than when the filesystem is considered stable and mature (tho certainly the rule applies even then, but try telling that to a mass-market user...). Additionally, since btrfs /is/ still stabilizing, we recommend that users run relatively new kernels, we best support the latest kernels in either of the current kernel series (thus 4.10 and 4.9 at present) or the mainline LTS series (thus 4.9 and 4.4 at present), and further recommend that users at least loosely follow the list in ordered to keep up with current btrfs developments and possible issues they may confront. That doesn't sound like a particularly good choice for a mass-market NAS product to me. Of course there's rockstor and others out there already shipping such products, but they're risking their reputation and the safety of their customer's data in the process, altho there's certainly a few customers out there with the time, desire and technical know-how to ensure the recommended backups and following current kernel and list, and that's exactly the sort of people you'll find already here. But that's not sufficiently mass-market to appeal to most vendors. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
