I've personally saturated 1Gbps links on multiple radosgw nodes on a large cluster, if I remember correctly, Yehuda has tested it up into the 7Gbps range with 10Gbps gear. Could you describe your clusters hardware and connectivity?
On Mon, Oct 14, 2013 at 3:34 AM, Chu Duc Minh <chu.ducm...@gmail.com> wrote: > Hi sorry, i missed this mail. > > > > During writes, does the CPU usage on your RadosGW node go way up? > No, CPU stay the same & very low (< 10%) > > When upload small files(300KB/file) over RadosGW: > - using 1 process: upload bandwidth ~ 3MB/s > - using 100 processes: upload bandwidth ~ 15MB/s > > When upload big files(3GB/file) over RadosGW: > - using 1 process: upload bandwidth ~ 70MB/s > (Therefore i don't upload big files using multi-processes any more :D) > > Maybe, RadosGW have a problem when write many smail files. Or it's a > problem of CEPH when simultaneously write many smail files into a bucket, > that already have millions files? > > > On Wed, Sep 25, 2013 at 7:24 PM, Mark Nelson <mark.nel...@inktank.com>wrote: > >> On 09/25/2013 02:49 AM, Chu Duc Minh wrote: >> >>> I have a CEPH cluster with 9 nodes (6 data nodes & 3 mon/mds nodes) >>> And i setup 4 separate nodes to test performance of Rados-GW: >>> - 2 node run Rados-GW >>> - 2 node run multi-process put file to [multi] Rados-GW >>> >>> Result: >>> a) When i use 1 RadosGW node & 1 upload-node, speed upload = 50MB/s >>> /upload-node, Rados-GW input/output speed = 50MB/s >>> >>> b) When i use 2 RadosGW node & 1 upload-node, speed upload = 50MB/s >>> /upload-node; each RadosGW have input/output = 25MB/s ==> sum >>> input/ouput of 2 Rados-GW = 50MB/s >>> >>> c) When i use 1 RadosGW node & 2 upload-node, speed upload = 25MB/s >>> /upload-node ==> sum output of 2 upload-node = 50MB/s, RadosGW have >>> input/output = 50MB/s >>> >>> d) When i use 2 RadosGW node & 2 upload-node, speed upload = 25MB/s >>> /upload-node ==> sum output of 2 upload-node = 50MB/s; each RadosGW have >>> input/output = 25MB/s ==> sum input/ouput of 2 Rados-GW = 50MB/s >>> >>> _*Problem*_: i can pass limit 50MB/s when put file over Rados-GW, >>> >>> regardless of the number Rados-GW nodes and upload-nodes. >>> When i use this CEPH cluster over librados (openstack/kvm), i can easily >>> achieve > 300MB/s >>> >>> I don't know why performance of RadosGW is so low. What's bottleneck? >>> >> >> During writes, does the CPU usage on your RadosGW node go way up? >> >> If this is a test cluster, you might want to try the wip-6286 build from >> our gitbuilder site. There is a fix that depending on the size of your >> objects, could have a big impact on performance. We're currently >> investigating some other radosgw performance issues as well, so stay tuned. >> :) >> >> Mark >> >> >> >>> Thank you very much! >>> >>> >>> >>> >>> ______________________________**_________________ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/**listinfo.cgi/ceph-users-ceph.**com<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >>> >>> >> ______________________________**_________________ >> ceph-users mailing list >> ceph-users@lists.ceph.com >> http://lists.ceph.com/**listinfo.cgi/ceph-users-ceph.**com<http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com> >> > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > -- Kyle
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
