On Sat, Mar 09, 2013 at 09:41:50PM -0800, Roger Binns wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 09/03/13 17:44, Hugo Mills wrote: > > You've got at least three independent parameters to the system in order > > to make that choice, though, and it's a fairly fuzzy decision problem. > > You've got: > > > > - Device redundancy - Storage overhead - Performance > > Overhead and performance aren't separate goals. More accurately the goal > is best performance given the devices available and constrained by redundancy. > > If I have 1GB of unique data and 10GB of underlying space available then > feel free to make 9 additional copies of each piece of data if that helps > performance. As I increase the unique data the overhead available will > decrease, but I doubt anyone has a goal of micromanaging overhead usage. > Why can't the filesystem just figure it out and do the best job available > given minimal constraints? > > > I definitely want to report the results in nCmSpP form, which tells you > > what it's actually done. The internal implementation, while not > > expressing the full gamut of possibilities, maps directly from the > > internal configuration to that form, and so it should at least be an > > allowable input for configuration (e.g. mkfs.btrfs and the restriper). > > Agreed on that for the micromanagers :-) > > > If you'd like to suggest a usable set of configuration axes [say, > > (redundancy, overhead) ], and a set of rules for converting those > > requirements to the internal representation, then there's no reason we > > can't add them as well in a later set of patches. > > The only constraints that matter are surviving N device failures, and data > not lost if at least N devices are still present. Under the hood the best > way of meeting those can be heuristically determined, and I'd expect > things like overhead to dynamically adjust as storage fills up or empties.
That's really not going to work happily -- you'd have to run the restriper in the background automatically as the device fills up. Given that this is going to end up rewriting *all* of the data on the FS, taking up storage bandwidth as it does it (and taking a hell of a long time to complete), I think this is a bit of a non-starter. Oh, and your performance will drop steadily over the months as it cranks down through the various options. If you want maximum storage (with some given redundancy), regardless of performance, then you might as well start with the parity-based levels and just leave it at that. Thinking about it, specifying a (redundancy, acceptable_wastage) pair is fairly pointless in controlling the performance levels, because it doesn't make the space/speed trade-off explicit. In fact, without actually doing long-term benchmarks on your hardware and workloads, it's going to be next-to-impossible to give any concrete figures, other than "X will be faster than Y". So, if you (as $sysadmin) actually care about performance, you'll have to benchmark your own system and test the various options anyway. If you don't care about performance and want as much storage as possible, you'll be using mS1P or mS2P (or possibly mS3P in the future). There's not much else a heuristic can do, without effectively exposing all the config options to the admin, in some obfuscated form. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- Great oxymorons of the world, no. 6: Mature Student ---
signature.asc
Description: Digital signature
