[ceph-users] Re: ceph fs perf stats output is empty
Each array has different types. You need to look at the below cephfs-top code and see how they are interpreted. [1] https://github.com/ceph/ceph/blob/main/src/tools/cephfs/top/cephfs-top#L66-L83 [2] https://github.com/ceph/ceph/blob/main/src/tools/cephfs/top/cephfs-top#L641-L714 [3] https://github.com/ceph/ceph/blob/main/src/tools/cephfs/top/cephfs-top#L110-L143 [4] https://github.com/ceph/ceph/blob/main/src/tools/cephfs/top/cephfs-top#L775-L823 On 12/06/23 14:23, Denis Polom wrote: Hi, yes, I've found that trick was I had to wait for about 15 sec to see the metrics. Now I can see some numbers. Are units there in miliseconds? And also I see 2 numbers reported - is the first value actual and second is delta? "client.4636": [ [ 924, 4 ], [ 0, 0 ], [ 10, 981017512 ], [ 0, 33484910 ], [ 136, 0 ], [ 1, 2 ], [ 2, 2 ], [ 1, 2 ], [ 0, 0 ], [ 302, 1266679808 ], [ 0, 0 ], [ 0, 0 ], [ 0, 36361015 ], [ 4205537661271535, 302 ], [ 0, 11161636 ], [ 190208004421472, 3 ] ] } }, Thx! On 6/12/23 06:36, Jos Collin wrote: Additionally, the first `ceph fs perf stats` output would be empty or outdated in most cases. You need to query a few times to get the latest values. So try `watch ceph fs perf stats`. On Mon, 12 Jun 2023 at 06:30, Xiubo Li wrote: On 6/10/23 05:35, Denis Polom wrote: > Hi > > I'm running latest Ceph Pacific 16.2.13 with Cephfs. I need to collect > performance stats per client, but getting empty list without any numbers > > I even run dd on client against mounted ceph fs, but output is only > like this: > > #> ceph fs perf stats 0 4638 192.168.121.1 > > {"version": 2, "global_counters": ["cap_hit", "read_latency", > "write_latency", "metadata_latency", "dentry_lease", "opened_files", > "pinned_icaps", " > opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", > "stdev_read_latency", "avg_write_latency", "stdev_write_latency", > "avg_metada > ta_latency", "stdev_metadata_latency"], "counters": [], > "client_metadata": {}, "global_metrics": {}, "metrics": > {"delayed_ranks": []}} > > Do I need to set some extra options? > Were you using the ceph-fuse/libcephfs user space clients ? if so you need to manually enable the 'client_collect_and_send_global_metrics' option, which is disabled in Ceph Pacific 16.2.13. While if you were using the kclient you need to make sure the 'disable_send_metrics' module parameter is 'false' for ceph.ko module. Thanks - Xiubo > Does it work for some of you guys? > > Thank you > > dp > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: ceph fs perf stats output is empty
Hi, yes, I've found that trick was I had to wait for about 15 sec to see the metrics. Now I can see some numbers. Are units there in miliseconds? And also I see 2 numbers reported - is the first value actual and second is delta? "client.4636": [ [ 924, 4 ], [ 0, 0 ], [ 10, 981017512 ], [ 0, 33484910 ], [ 136, 0 ], [ 1, 2 ], [ 2, 2 ], [ 1, 2 ], [ 0, 0 ], [ 302, 1266679808 ], [ 0, 0 ], [ 0, 0 ], [ 0, 36361015 ], [ 4205537661271535, 302 ], [ 0, 11161636 ], [ 190208004421472, 3 ] ] } }, Thx! On 6/12/23 06:36, Jos Collin wrote: Additionally, the first `ceph fs perf stats` output would be empty or outdated in most cases. You need to query a few times to get the latest values. So try `watch ceph fs perf stats`. On Mon, 12 Jun 2023 at 06:30, Xiubo Li wrote: On 6/10/23 05:35, Denis Polom wrote: > Hi > > I'm running latest Ceph Pacific 16.2.13 with Cephfs. I need to collect > performance stats per client, but getting empty list without any numbers > > I even run dd on client against mounted ceph fs, but output is only > like this: > > #> ceph fs perf stats 0 4638 192.168.121.1 > > {"version": 2, "global_counters": ["cap_hit", "read_latency", > "write_latency", "metadata_latency", "dentry_lease", "opened_files", > "pinned_icaps", " > opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", > "stdev_read_latency", "avg_write_latency", "stdev_write_latency", > "avg_metada > ta_latency", "stdev_metadata_latency"], "counters": [], > "client_metadata": {}, "global_metrics": {}, "metrics": > {"delayed_ranks": []}} > > Do I need to set some extra options? > Were you using the ceph-fuse/libcephfs user space clients ? if so you need to manually enable the 'client_collect_and_send_global_metrics' option, which is disabled in Ceph Pacific 16.2.13. While if you were using the kclient you need to make sure the 'disable_send_metrics' module parameter is 'false' for ceph.ko module. Thanks - Xiubo > Does it work for some of you guys? > > Thank you > > dp > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: ceph fs perf stats output is empty
Additionally, the first `ceph fs perf stats` output would be empty or outdated in most cases. You need to query a few times to get the latest values. So try `watch ceph fs perf stats`. On Mon, 12 Jun 2023 at 06:30, Xiubo Li wrote: > > On 6/10/23 05:35, Denis Polom wrote: > > Hi > > > > I'm running latest Ceph Pacific 16.2.13 with Cephfs. I need to collect > > performance stats per client, but getting empty list without any numbers > > > > I even run dd on client against mounted ceph fs, but output is only > > like this: > > > > #> ceph fs perf stats 0 4638 192.168.121.1 > > > > {"version": 2, "global_counters": ["cap_hit", "read_latency", > > "write_latency", "metadata_latency", "dentry_lease", "opened_files", > > "pinned_icaps", " > > opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", > > "stdev_read_latency", "avg_write_latency", "stdev_write_latency", > > "avg_metada > > ta_latency", "stdev_metadata_latency"], "counters": [], > > "client_metadata": {}, "global_metrics": {}, "metrics": > > {"delayed_ranks": []}} > > > > Do I need to set some extra options? > > > Were you using the ceph-fuse/libcephfs user space clients ? if so you > need to manually enable the 'client_collect_and_send_global_metrics' > option, which is disabled in Ceph Pacific 16.2.13. > > While if you were using the kclient you need to make sure the > 'disable_send_metrics' module parameter is 'false' for ceph.ko module. > > Thanks > > - Xiubo > > > > Does it work for some of you guys? > > > > Thank you > > > > dp > > ___ > > ceph-users mailing list -- ceph-users@ceph.io > > To unsubscribe send an email to ceph-users-le...@ceph.io > > > ___ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > > ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
[ceph-users] Re: ceph fs perf stats output is empty
On 6/10/23 05:35, Denis Polom wrote: Hi I'm running latest Ceph Pacific 16.2.13 with Cephfs. I need to collect performance stats per client, but getting empty list without any numbers I even run dd on client against mounted ceph fs, but output is only like this: #> ceph fs perf stats 0 4638 192.168.121.1 {"version": 2, "global_counters": ["cap_hit", "read_latency", "write_latency", "metadata_latency", "dentry_lease", "opened_files", "pinned_icaps", " opened_inodes", "read_io_sizes", "write_io_sizes", "avg_read_latency", "stdev_read_latency", "avg_write_latency", "stdev_write_latency", "avg_metada ta_latency", "stdev_metadata_latency"], "counters": [], "client_metadata": {}, "global_metrics": {}, "metrics": {"delayed_ranks": []}} Do I need to set some extra options? Were you using the ceph-fuse/libcephfs user space clients ? if so you need to manually enable the 'client_collect_and_send_global_metrics' option, which is disabled in Ceph Pacific 16.2.13. While if you were using the kclient you need to make sure the 'disable_send_metrics' module parameter is 'false' for ceph.ko module. Thanks - Xiubo Does it work for some of you guys? Thank you dp ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io