Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 +1 - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Thu, Jun 25, 2015 at 10:39 AM, Alexandre DERUMIER wrote: Thanks Mark ! - Mail original - De: Mark Nelson À: aderumier , Robert

RE: deleting objects from a pool

2015-06-25 Thread Podoski, Igor
Hi David, You're right, now I see adding --run-name will clean all benchmark data from specified namespace, so you can run command only once. rados -p poolname -N namespace cleanup --prefix --run-name Regards, Igor. -Original Message- From: David Zafman [mailto:dzaf...@redhat.com]

Re: [PATCH] libceph: Fix ceph_tcp_sendpage()'s more boolean usage

2015-06-25 Thread Alex Elder
On 06/25/2015 11:56 AM, Benoît Canet wrote: Spotted while hunting http://tracker.ceph.com/issues/10905. From struct ceph_msg_data_cursor in include/linux/ceph/messenger.h: boollast_piece; /* current is last piece */ In ceph_msg_data_next(): *last_piece = cursor-last_piece; A call

Re: [PATCH 3/3] rbd: queue_depth map option

2015-06-25 Thread Ilya Dryomov
On Thu, Jun 25, 2015 at 5:24 PM, Alex Elder el...@ieee.org wrote: On 06/25/2015 04:11 AM, Ilya Dryomov wrote: nr_requests (/sys/block/rbdid/queue/nr_requests) is pretty much irrelevant in blk-mq case because each driver sets its own max depth that it can handle and that's the number of tags

Re: [PATCH] rbd: bump queue_max_segments

2015-06-25 Thread Alex Elder
On 06/25/2015 04:01 AM, Ilya Dryomov wrote: The default queue_limits::max_segments value (BLK_MAX_SEGMENTS = 128) unnecessarily limits bio sizes to 512k (assuming 4k pages). rbd, being a virtual block device, doesn't have any restrictions on the number of physical segments, so bump max_segments

Re: [PATCH 3/3] rbd: queue_depth map option

2015-06-25 Thread Alex Elder
On 06/25/2015 04:11 AM, Ilya Dryomov wrote: nr_requests (/sys/block/rbdid/queue/nr_requests) is pretty much irrelevant in blk-mq case because each driver sets its own max depth that it can handle and that's the number of tags that gets preallocated on setup. Users can't increase queue depth

Re: Ceph hard lock Hammer 9.2

2015-06-25 Thread Barclay Jameson
If I get it to happen again I will send you the kernel message. Thanks again Zheng! On Wed, Jun 24, 2015 at 8:48 AM, Yan, Zheng uker...@gmail.com wrote: Could you please run echo 1 /proc/sys/kernel/sysrq; echo t /proc/sysrq-trigger when this warning happens again. then send the kernel

Re: [PATCH] Avoid holding the zero page on slab init failure

2015-06-25 Thread Alex Elder
On 06/24/2015 08:27 PM, Benoît Canet wrote: Spotted by visual inspection. Applies on libceph: Remove spurious kunmap() of the zero page. Benoît Canet (1): libceph: Avoid holding the zero page on ceph_msgr_slab_init errors net/ceph/messenger.c | 10 +- 1 file changed, 5

Re: [PATCH 1/3] rbd: terminate rbd_opts_tokens with Opt_err

2015-06-25 Thread Alex Elder
On 06/25/2015 04:11 AM, Ilya Dryomov wrote: Also nuke useless Opt_last_bool and don't break lines unnecessarily. Signed-off-by: Ilya Dryomov idryo...@gmail.com Good idea. Reviewed-by: Alex Elder el...@linaro.org --- drivers/block/rbd.c | 24 1 file changed, 8

Re: [PATCH 2/3] rbd: store rbd_options in rbd_device

2015-06-25 Thread Alex Elder
On 06/25/2015 04:11 AM, Ilya Dryomov wrote: Signed-off-by: Ilya Dryomov idryo...@gmail.com Now that you need it when initializing the disk, this makes sense. Reviewed-by: Alex Elder el...@linaro.org --- drivers/block/rbd.c | 18 +++--- 1 file changed, 11 insertions(+), 7

Re: [PATCH] libceph: Avoid holding the zero page on ceph_msgr_slab_init errors

2015-06-25 Thread Alex Elder
On 06/24/2015 08:27 PM, Benoît Canet wrote: ceph_msgr_slab_init may fail due to a temporary ENOMEM. Looks good. Delay a bit the initialization of zero_page in ceph_msgr_init and reorder it's cleanup in _ceph_msgr_exit for readability sake. I'd say it's not readability, but a proper

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Alexandre DERUMIER
Thanks Mark ! - Mail original - De: Mark Nelson mnel...@redhat.com À: aderumier aderum...@odiso.com, Robert LeBlanc rob...@leblancnet.us Cc: ceph-devel ceph-devel@vger.kernel.org Envoyé: Jeudi 25 Juin 2015 18:36:45 Objet: Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON! Hi Guys,

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Robert LeBlanc
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Any update on the recorded sessions? Last I see is 10 Jun 2015 in the Etherpad. Thanks, - Robert LeBlanc GPG Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 On Wed, Jun 24, 2015 at 7:28 AM, Mark Nelson wrote: 8AM

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Alexandre DERUMIER
I would like to have them too ;) I have missed the yesterday session, I would like to have infos about this Fujitsu presenting on bufferlist tuning - about 2X savings in overall CPU Time with new code. - Mail original - De: Robert LeBlanc rob...@leblancnet.us À: Mark Nelson

[PATCH] libceph: Fix ceph_tcp_sendpage()'s more boolean usage

2015-06-25 Thread Benoît Canet
Spotted while hunting http://tracker.ceph.com/issues/10905. From struct ceph_msg_data_cursor in include/linux/ceph/messenger.h: boollast_piece; /* current is last piece */ In ceph_msg_data_next(): *last_piece = cursor-last_piece; A call to ceph_msg_data_next() is followed by: ret =

Re: [PATCH] libceph: Fix ceph_tcp_sendpage()'s more boolean usage

2015-06-25 Thread Ilya Dryomov
On Thu, Jun 25, 2015 at 7:56 PM, Benoît Canet benoit.ca...@nodalink.com wrote: Spotted while hunting http://tracker.ceph.com/issues/10905. From struct ceph_msg_data_cursor in include/linux/ceph/messenger.h: boollast_piece; /* current is last piece */ In ceph_msg_data_next():

Re: 06/24/2015 Weekly Ceph Performance Meeting IS ON!

2015-06-25 Thread Mark Nelson
Hi Guys, I've updated the etherpad with links to the new recordings. Sorry these get backlogged. It takes a little while for the recording to become available and then I have to manually go in and mark each of them public and available for download. There doesn't seem to be any way to set

RE: deleting objects from a pool

2015-06-25 Thread Deneau, Tom
Igor -- Good command to know, but this is still very slow on an erasure pool. For example, on my cluster it took 10 seconds with rados bench to write 10,000 40K size objects to an ecpool. And it took almost 6 minutes to delete them using the command below. -- Tom -Original Message-

[PATCH v2] libceph: Avoid holding the zero page on ceph_msgr_slab_init errors

2015-06-25 Thread Benoît Canet
ceph_msgr_slab_init may fail due to a temporary ENOMEM. Delay a bit the initialization of zero_page in ceph_msgr_init and reorder it's cleanup in _ceph_msgr_exit so it's done in reverse order of setup. BUG_ON() will not suffer to be postponed in case it is triggered. Signed-off-by: Benoît Canet

Re: Inline dedup/compression

2015-06-25 Thread Benoît Canet
Le Ven 26 juin 2015, à 00:01, James (Fei) Liu-SSI a écrit : Hi Cephers, It is not easy to ask when Ceph is going to support inline dedup/compression across OSDs in RADOS. disclamer: I am not a Cepher. This would mean some kind of distributed key value store that is fast enough to

Re: Inline dedup/compression

2015-06-25 Thread Haomai Wang
On Fri, Jun 26, 2015 at 6:01 AM, James (Fei) Liu-SSI james@ssi.samsung.com wrote: Hi Cephers, It is not easy to ask when Ceph is going to support inline dedup/compression across OSDs in RADOS because it is not easy task and answered. Ceph is providing replication and EC for

Re: deleting objects from a pool

2015-06-25 Thread David Zafman
If you have rados bench data around, you'll need to run cleanup a second time because the first time the benchmark_last_metadata object will be consulted to find what objects to remove. Also, using cleanup this way will only remove objects from the default namespace unless a namespace is

RE: deleting objects from a pool

2015-06-25 Thread Podoski, Igor
Hi, It appears, that cleanup can be used as a purge: rados -p poolname cleanup --prefix Regards, Igor. -Original Message- From: ceph-devel-ow...@vger.kernel.org [mailto:ceph-devel-ow...@vger.kernel.org] On Behalf Of Deneau, Tom Sent: Wednesday, June 24, 2015 10:22 PM To: Dałek,

Re: [PATCH] libceph: Remove spurious kunmap() of the zero page

2015-06-25 Thread Ilya Dryomov
On Thu, Jun 25, 2015 at 3:35 AM, Alex Elder el...@ieee.org wrote: On 06/24/2015 04:18 PM, Benoît Canet wrote: ceph_tcp_sendpage already does the work of mapping/unmapping the zero page if needed. Signed-off-by: Benoît Canet benoit.ca...@nodalink.com This looks good. Reviewed-by: Alex

[PATCH 3/3] rbd: queue_depth map option

2015-06-25 Thread Ilya Dryomov
nr_requests (/sys/block/rbdid/queue/nr_requests) is pretty much irrelevant in blk-mq case because each driver sets its own max depth that it can handle and that's the number of tags that gets preallocated on setup. Users can't increase queue depth beyond that value via writing to nr_requests.

[PATCH] rbd: bump queue_max_segments

2015-06-25 Thread Ilya Dryomov
The default queue_limits::max_segments value (BLK_MAX_SEGMENTS = 128) unnecessarily limits bio sizes to 512k (assuming 4k pages). rbd, being a virtual block device, doesn't have any restrictions on the number of physical segments, so bump max_segments to max_hw_sectors, in theory allowing a

[PATCH 0/3] rbd: queue_depth map option

2015-06-25 Thread Ilya Dryomov
Hi, See 3/3. I'll patch rbd cli tool once this is in. Thanks, Ilya Ilya Dryomov (3): rbd: terminate rbd_opts_tokens with Opt_err rbd: store rbd_options in rbd_device rbd: queue_depth map option drivers/block/rbd.c | 59

[PATCH 2/3] rbd: store rbd_options in rbd_device

2015-06-25 Thread Ilya Dryomov
Signed-off-by: Ilya Dryomov idryo...@gmail.com --- drivers/block/rbd.c | 18 +++--- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index 4de8c9167c4b..e502bce02d2c 100644 --- a/drivers/block/rbd.c +++ b/drivers/block/rbd.c @@

[PATCH 1/3] rbd: terminate rbd_opts_tokens with Opt_err

2015-06-25 Thread Ilya Dryomov
Also nuke useless Opt_last_bool and don't break lines unnecessarily. Signed-off-by: Ilya Dryomov idryo...@gmail.com --- drivers/block/rbd.c | 24 1 file changed, 8 insertions(+), 16 deletions(-) diff --git a/drivers/block/rbd.c b/drivers/block/rbd.c index