Re: Ceph-fuse single read limitation?‏

[Yan, Zheng]

> On Nov 21, 2015, at 10:12, Z Zhang  wrote:
> 
> Hi Guys,
> 
> Now we have a very small cluster with 3 OSDs but using 40Gb NIC. We use 
> ceph-fuse as cephfs client and enable readahead, but testing single reading a 
> large file from cephfs via fio, dd or cp can only achieve ~70+MB/s, even if 
> fio or dd's block size is set to 1MB or 4MB.
> 
> From the ceph client log, we found each read request's size (ll_read) is 
> limited to 128KB, which should be the limitation of kernel fuse's 
> FUSE_MAX_PAGES_PER_REQ = 32. We may try to increase this value to see the 
> performance difference, but are there other options we can try to increase 
> single read performance?
> 

try setting client_readahead_max_bytes to 4M

Yan, Zheng

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: why my cluster become unavailable

[hzwulibin]

Hi, Sage

Thanks! Will try it when next testing!

--   
hzwulibin
2015-11-23

-
发件人：Sage Weil 
发送日期：2015-11-22 01:49
收件人：Haomai Wang
抄送：Libin Wu,ceph-devel
主题：Re: why my cluster become unavailable

On Sun, 22 Nov 2015, Haomai Wang wrote:
> On Thu, Nov 19, 2015 at 11:26 PM, Libin Wu  wrote:
> > Hi, cepher
> >
> > I have a cluster of 6 OSD server, every server has 8 OSDs.
> >
> > I out 4 OSDs on every server, then my client io is blocking.
> >
> > I reboot my client and then create a new rbd device, but the new
> > device also can't write io.
> >
> > Yeah, i understand that some data may lost as threee replicas of some
> > object were lost, but why the cluster become unavailable?
> >
> > There 80 incomplete pg and 4 down+incomplete pg.
> >
> > Any solution i could solve the problem?
> 
> Yes, if you doesn't have a special crushmap to control the data
> replcement policy, pg will lack of necessary metadata to boot. If need
> to readd outed osds or force remove pg which is incomplete(hope it's
> just a test).

Is min_size 2 or 1?  Reducing it to 1 will generally clear some of the 
incomplete pgs.  Just remember to raise it back to 2 after the cluster 
recovers.

sage

N�r��yb�X��ǧv�^�)޺{.n�+���z�]z���{ay�ʇڙ�,j��f���h���z��w���
���j:+v���w�j�mzZ+�ݢj"��!�i

[no subject]

[Dong Wu]

subscribe ceph-devel
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html