On Thu, Jan 31, 2019 at 10:01 AM Xavi Hernandez <xhernan...@redhat.com> wrote:
> Hi, > > I've been doing some tests with the global thread pool [1], and I've > observed one important thing: > > Since this new thread pool has very low contention (apparently), it > exposes other problems when the number of threads grows. What I've seen is > that some workloads use all available threads on bricks to do I/O, causing > avgload to grow rapidly and saturating the machine (or it seems so), which > really makes everything slower. Reducing the maximum number of threads > improves performance actually. Other workloads, though, do little I/O > (probably most is locking or smallfile operations). In this case limiting > the number of threads to a small value causes a performance reduction. To > increase performance we need more threads. > > So this is making me thing that maybe we should implement some sort of I/O > queue with a maximum I/O depth for each brick (or disk if bricks share same > disk). This way we can limit the amount of requests physically accessing > the underlying FS concurrently, without actually limiting the number of > threads that can be doing other things on each brick. I think this could > improve performance. > Perhaps we could throttle both aspects - number of I/O requests per disk and the number of threads too? That way we will have the ability to behave well when there is bursty I/O to the same disk and when there are multiple concurrent requests to different disks. Do you have a reason to not limit the number of threads? > Maybe this approach could also be useful in client side, but I think it's > not so critical there. > Agree, rate limiting on the server side would be more appropriate. Thanks, Vijay
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel
