On 10/15/2015 02:57 AM, Dan Egli wrote: > Okay. Now my lack of experience in the last ten years is coming back to > haunt me. Perhaps this is a simple question, and if so I'm sorry, but I > have to ask. What the heck is an extent? Realize I've never even seen ext4. > I was just getting used to a journaling ext2 (aka ext3) when I was forced > to stop using Linux for some time. I'm only now getting back into things.
An extent is a contiguous area of storage on the disk, ensuring that files are kept in one place where possible, instead of spread across multiple areas of the disk. As long as most files are able to be written to extents, you'll have minimal fragmentation. > Frankly, I don't care if I use Ext4, XFS, Btrfs, or something else on this > file system I'll be setting up. But I do expect some heavy fragmentation > due to multiple downloaders functioning at once (I would not be surprised > to see six or seven files being written to at once). My only requirements > are that it be a fast filesystem, and that there be some way to reduce the > fragmentation to a minimum so that for later transfers the read speed is as > close to the max combined speed for the multiple HDDs in the volume as > possible. And If I was to use XFS or Btrfs, I'd probably want to look into > Data Deduplication as well, but that's not a guarantee. For large multi-media files, I don't know that dedup is going to be that necessary. Another fairly good option is ZFS. ZFS On Linux is a fairly mature project now and there's talk of Ubuntu shipping the ZoL binary kernel module in the future so you won't have to compile it yourself after every kernel update. Supposedly the lawyers feel this does not violate the GPL, though the kernel modules do have to compile against kernel APIs). Would be excellent if they do figure out a way to do it. There's a good reason to use BtrFS or ZFS. That's bitrot. With huge file systems full of rarely accessed but important data, it's highly possible that the odd bit will degrade and flip here and there. Maybe that would matter, maybe it wouldn't. But Ext4 has no means of verifying files. ZFS verifies every block, and can correct bit errors, allowing you to copy the file and ensure it's still pristine. BtrFS does something similar I understand. One bit may not matter in a text file particularly, but it can make a JPG have rendering issues. http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/ > I've certainly heard of XFS before, and it sounds like it might suit the > bill quite nicely. But if I do use it, how do I setup these "extents" you > speak of? And what is a good size for the extent? This volume will store > some programs, but mostly multi-media files (approx 10% programs, 90% > multi-media, of which 95% will be video files). You don't have to set anything up. Management of extents is part of the file systems that support it. Just do a mkfs.ext4 (or xfs or btrfs) and you have it. XFS is the default file system in Red Hat EL 7 as it can scale beyond ext4 volume sizes. I used XFS some years ago, but I found that power outages could sometimes corrupt files that were open. I'm not sure if this is still the case with XFS, but I recommend a UPS in any case. Theoretically ZFS can be interrupted at any time and the file system is always consistent, although any non-committed blocks would be lost. It's the only file system out there that has this property at all times. BtrFS is close, but there are one or two circumstances where data loss is still possible. /* PLUG: http://plug.org, #utah on irc.freenode.net Unsubscribe: http://plug.org/mailman/options/plug Don't fear the penguin. */
