> -----Original Message-----
> From: Gregory Farnum [mailto:g...@gregs42.com]
> Sent: Thursday, May 28, 2015 6:18 PM
> To: Deneau, Tom
> Cc: ceph-devel
> Subject: Re: rados bench throughput with no disk or network activity
>
> On Thu, May 28, 2015 at 4:09 PM, Deneau, Tom <tom.den...@amd.com> wrote:
> > I've noticed that
> > * with a single node cluster with 4 osds
> > * and running rados bench rand on that same node so no network traffic
> > * with a number of objects small enough so that everything is in
> > the cache so no disk traffic
> >
> > we still peak out at about 1600 MB/sec.
> >
> > And the cpu is 40% idle. (and a good chunk of the cpu activity is the
> > rados benchmark itself)
> >
> > What is likely causing the throttling here?
>
> Well, rados bench itself is essentially single-threaded, so if it's using
> 100% CPU that's probably the bottleneck you're hitting.
>
> Otherwise, by default it will limit itself to 100MB of outstanding IO
> (there's an objecter config value you can change for this; it's been
> discussed recently) and that might not be enough given the latencies of
> hopping packets across different CPUs, and the OSDs have a slightly-
> embarrassing amount of CPU computation and thread hopping they have to
> perform on every op (around half a millisecond's worth on each read, I
> think?).
> -Gerg
Right. I was involved in the objecter config discussion :) and
I have set the limits higher. And this 1600 MB/sec limit seems to be
the same whatever the size of the objects.
rados bench is using about 30% of the cpu and the total cpu usage is about 60%
(the rest being mostly from the 4 osds).
Hmm, I just tried running 4 copies of rados bench rand, and I can get a little
bit
higher combined totals, but not much higher maybe 1800 MB/sec.
-- Tom
N�����r��y����b�X��ǧv�^�){.n�+���z�]z���{ay� ʇڙ�,j��f���h���z� �w���
���j:+v���w�j�m��������zZ+�����ݢj"��!�i
