On Thu, Dec 18, 2014 at 8:44 PM, Sean Sullivan <seapasu...@uchicago.edu> wrote: > Thanks for the reply Gegory, > > Sorry if this is in the wrong direction or something. Maybe I do not > understand > > To test uploads I either use bash time and either python-swiftclient or boto > key.set_contents_from_filename to the radosgw. I was unaware that radosgw > had any type of throttle settings in the configuration (I can't seem to find > any either). As for rbd mounts I test by creating a 1TB mount and writing a > file to it through time+cp or dd. Not the most accurate test but I think > should be good enough as a quick functionality test. So for writes, it's > more for functionality than performance. I would think a basic functionality > test should yield more than 8mb/s though. > > As for checking admin sockets: I have actually, I set the 3rd gateways > debug_civetweb to 10 as well as debug_rgw to 5 but I still do not see > anything that stands out. The snippet of the log I pasted has these values > set. I did the same for an osd that is marked as slow (1112). All I can see > in the log for the osd are ticks and heartbeat responses though, nothing > that shows any issues. Finally I did it for the primary monitor node to see > if I would see anything there with debug_mon set to 5 > (http://pastebin.com/hhnaFac1). I do not really see anything that would > stand out as a failure (like a fault or timeout error). > > What kind of throttler limits do you mean? I didn't/don't see any mention of > rgw throttler limits in the ceph.com docs or admin socket just osd/ > filesystem throttle like inode/flusher limits, do you mean these? I have not > messed with these limits yet on this cluster, do you think it would help?
The admin socket is different from the logs: http://ceph.com/docs/master/rados/troubleshooting/troubleshooting-osd/#admin-socket You should run the "help" command against it to see what you can do, but in particular it lets you see the status of running operations and history of slow ones, in addition to "perfcounters" which will let you see things like "throttler" counts. If any of those are persistently full on your OSDs at the same time as you have operations which seem to be stuck waiting to get in, that would be a hint. :) -Greg _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
