On 07/27/2014 11:21 PM, Nick Krause wrote: > On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn > <ahferro...@gmail.com> wrote: >> On 07/27/2014 04:47 PM, Nick Krause wrote: >>> This may be a bad idea , but compression in brtfs seems to be only >>> using one core to compress. >>> Depending on the CPU used and the amount of cores in the CPU we can >>> make this much faster >>> with multiple cores. This seems bad by my reading at least I would >>> recommend for writing compression >>> we write a function to use a certain amount of cores based on the load >>> of the system's CPU not using >>> more then 75% of the system's CPU resources as my system when idle has >>> never needed more >>> then one core of my i5 2500k to run when with interrupts for opening >>> eclipse are running. For reading >>> compression on good core seems fine to me as testing other compression >>> software for reads , it's >>> way less CPU intensive. >>> Cheers Nick >> We would probably get a bigger benefit from taking an approach like >> SquashFS has recently added, that is, allowing multi-threaded >> decompression fro reads, and decompressing directly into the pagecache. >> Such an approach would likely make zlib compression much more scalable >> on large systems. >> >> > > Austin, > That seems better then my idea as you seem to be more up to date on > brtfs devolopment. > If you and the other developers of brtfs are interested in adding this > as a feature please let > me known as I would like to help improve brtfs as the file system as > an idea is great just > seems like it needs a lot of work :). > Nick I wouldn't say that I am a BTRFS developer (power user maybe?), but I would definitely say that parallelizing compression on writes would be a good idea too (especially for things like lz4, which IIRC is either in 3.16 or in the queue for 3.17). Both options would be a lot of work, but almost any performance optimization would. I would almost say that it would provide a bigger performance improvement to get BTRFS to intelligently stripe reads and writes (at the moment, any given worker thread only dispatches one write or read to a single device at a time, and any given write() or read() syscall gets handled by only one worker).
smime.p7s
Description: S/MIME Cryptographic Signature