On Tue, Feb 5, 2019 at 10:57 PM Xavi Hernandez <xhernan...@redhat.com> wrote:
> On Wed, Feb 6, 2019 at 7:00 AM Poornima Gurusiddaiah <pguru...@redhat.com> > wrote: > >> >> >> On Tue, Feb 5, 2019, 10:53 PM Xavi Hernandez <xhernan...@redhat.com >> wrote: >> >>> On Fri, Feb 1, 2019 at 1:51 PM Xavi Hernandez <xhernan...@redhat.com> >>> wrote: >>> >>>> On Fri, Feb 1, 2019 at 1:25 PM Poornima Gurusiddaiah < >>>> pguru...@redhat.com> wrote: >>>> >>>>> Can the threads be categorised to do certain kinds of fops? >>>>> >>>> >>>> Could be, but creating multiple thread groups for different tasks is >>>> generally bad because many times you end up with lots of idle threads which >>>> waste resources and could increase contention. I think we should only >>>> differentiate threads if it's absolutely necessary. >>>> >>>> >>>>> Read/write affinitise to certain set of threads, the other metadata >>>>> fops to other set of threads. So we limit the read/write threads and not >>>>> the metadata threads? Also if aio is enabled in the backend the threads >>>>> will not be blocked on disk IO right? >>>>> >>>> >>>> If we don't block the thread but we don't prevent more requests to go >>>> to the disk, then we'll probably have the same problem. Anyway, I'll try to >>>> run some tests with AIO to see if anything changes. >>>> >>> >>> I've run some simple tests with AIO enabled and results are not good. A >>> simple dd takes >25% more time. Multiple parallel dd take 35% more time to >>> complete. >>> >> >> >> Thank you. That is strange! Had few questions, what tests are you running >> for measuring the io-threads performance(not particularly aoi)? is it dd >> from multiple clients? >> > > Yes, it's a bit strange. What I see is that many threads from the thread > pool are active but using very little CPU. I also see an AIO thread for > each brick, but its CPU usage is not big either. Wait time is always 0 (I > think this is a side effect of AIO activity). However system load grows > very high. I've seen around 50, while on the normal test without AIO it's > stays around 20-25. > > Right now I'm running the tests on a single machine (no real network > communication) using an NVMe disk as storage. I use a single mount point. > The tests I'm running are these: > > - Single dd, 128 GiB, blocks of 1MiB > - 16 parallel dd, 8 GiB per dd, blocks of 1MiB > - fio in sequential write mode, direct I/O, blocks of 128k, 16 > threads, 8GiB per file > - fio in sequential read mode, direct I/O, blocks of 128k, 16 threads, > 8GiB per file > - fio in random write mode, direct I/O, blocks of 128k, 16 threads, > 8GiB per file > - fio in random read mode, direct I/O, blocks of 128k, 16 threads, > 8GiB per file > - smallfile create, 16 threads, 256 files per thread, 32 MiB per file > (with one brick down, for the following test) > - self-heal of an entire brick (from the previous smallfile test) > - pgbench init phase with scale 100 > > I run all these tests for a replica 3 volume and a disperse 4+2 volume. > Are these performance results available somewhere? I am quite curious to understand the performance gains on NVMe! Thanks, Vijay
_______________________________________________ Gluster-devel mailing list Gluster-devel@gluster.org https://lists.gluster.org/mailman/listinfo/gluster-devel