Re: [lustre-discuss] User find out OST configuration

2023-01-21 Thread Andreas Dilger via lustre-discuss 
Hi Anna,
Beyond the number and size of OSTs and MDTs there isn't much information about 
the underlying storage available on the client.

The "lfs df -v" command will print a "f" at the end for flash (non-rotational) 
devices, if the storage is properly configured.  The "osc*.imports " parameter 
file will contain some information about the grant_block_size that can be used 
to distinguish ldiskfs (4096) vs. zfs backends (131072 or 1048576).

The size of the disks can often be inferred from 1/8 of the total OST size for 
standard 8+2 RAID configs, but this may vary and no actual device-level metrics 
are available on the client.

Even on the server, Lustre itself doesn't know or care much about the 
underlying storage devices beyond (non-)rotational state, so we don't track any 
of that.

Cheers, Andreas

On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss 
 wrote:

 Hi,

is it possible for a user (no root, so ssh to server) to find out the 
configuration of an OST?
How many devices are there in one OST 'pool' (for both ldiskfs and ZFS) and 
even which type of devices they are (nvme, ssd, hdd)? Maybe even speeds and 
raid-levels?

Additionally, how can a user find out the mapping of all available OSTs to OSSs 
easily?

Thanks
Anna

--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] User find out OST configuration

2023-01-23 Thread Anna Fuchs via lustre-discuss 

Thanks!

Is it planned to introduce some metric propagation to the user?
For advanced users who are benchmarking stuff on remote systems it 
remains unclear which performance to expect if they can not access 
underlaying hardware metrics.
Sure, they can ask the admin to share the config, but it might be more 
convenient to be able to look it up, maybe.


Additionally: if I try to find out the stripe location (lfs gestripe) 
and map this information to OST-specs (lctl get_param 
osc.*.*ost_conn_uuid), to find out how many different servers and 
networks are involved, the obdidx seems to be in dec-format, but the OST 
index in connections list is hex, which is not always obvious.

Is there a way to display it both in dec or both in hex?

Are there generally any tools for doing similar things?
We plan a student project for building kind of GUI for visualizing 
stripings and mappings, so I would try to avoid reinventing the wheel.


Thank you.

Best regards
Anna

Am 21.01.2023 um 17:08 schrieb Andreas Dilger:

Hi Anna,
Beyond the number and size of OSTs and MDTs there isn't much 
information about the underlying storage available on the client.


The "lfs df -v" command will print a "f" at the end for flash 
(non-rotational) devices, if the storage is properly configured.  The 
"osc*.imports " parameter file will contain some information about the 
grant_block_size that can be used to distinguish ldiskfs (4096) vs. 
zfs backends (131072 or 1048576).


The size of the disks can often be inferred from 1/8 of the total OST 
size for standard 8+2 RAID configs, but this may vary and no actual 
device-level metrics are available on the client.


Even on the server, Lustre itself doesn't know or care much about the 
underlying storage devices beyond (non-)rotational state, so we don't 
track any of that.


Cheers, Andreas

On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss 
 wrote:


 Hi,

is it possible for a user (no root, so ssh to server) to find out the 
configuration of an OST?
How many devices are there in one OST 'pool' (for both ldiskfs and 
ZFS) and even which type of devices they are (nvme, ssd, hdd)? Maybe 
even speeds and raid-levels?


Additionally, how can a user find out the mapping of all available 
OSTs to OSSs easily?


Thanks
Anna
--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] User find out OST configuration

2023-01-23 Thread Laura Hild via lustre-discuss 
> Additionally, how can a user find out the mapping of all available OSTs to 
> OSSs easily?

I've used variations on

  grep -Fe failover_nids: -e current_connection: /proc/fs/lustre/*c/*/import
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org

Re: [lustre-discuss] User find out OST configuration

2023-01-23 Thread Andreas Dilger via lustre-discuss 
On Jan 23, 2023, at 10:01, Anna Fuchs 
mailto:anna.fu...@uni-hamburg.de>> wrote:

Thanks!

Is it planned to introduce some metric propagation to the user?
For advanced users who are benchmarking stuff on remote systems it remains 
unclear which performance to expect if they can not access underlaying hardware 
metrics.  Sure, they can ask the admin to share the config, but it might be 
more convenient to be able to look it up, maybe.

Yes,  there is a longstanding open ticket to export some server statistics to 
the clients - https://jira.whamcloud.com/browse/LU-7880 "add performance 
statistics to obd_statfs".  As the summary describes, this would export basic 
performance stats for each OST/MDT device to the client for current/peak x 
read/write x IOPS/bandwidth.  There are a number of potential uses for this, 
such as client/MDS selection of OSTs based on bandwidth/IOPS (beyond just 
"rotational" or "non-rotational"), userspace applications/libraries using it to 
determine if the OSTs are less busy (e.g. when to checkpoint), etc.

There aren't any plans to be able to export the storage "config" (e.g. RAID 
geometry) via Lustre since this is often opaque even on the server, and doesn't 
have any use on the client.  There was a discussion in the context of IO500 to 
write a script for collecting storage system configuration metadata for Lustre 
and other filesystems (e.g. list of OST/MDT devices, list of SCSI devices, PCI 
devices, CPU, RAM, network interfaces, etc.)

Additionally: if I try to find out the stripe location (lfs getstripe) and map 
this information to OST-specs (lctl get_param osc.*.*ost_conn_uuid), to find 
out how many different servers and networks are involved, the obdidx seems to 
be in dec-format, but the OST index in connections list is hex, which is not 
always obvious.  Is there a way to display it both in dec or both in hex?

There isn't currently an option for "lfs getstripe" to print in hex, but it 
would be possible to add a "--hex" option to print the fields in hex.  I've 
filed https://jira.whamcloud.com/browse/LU-16503 for tracking this issue.  I 
don't think it would be terribly complex to implement, but I also don't know if 
anyone is available to do this work right now.

Are there generally any tools for doing similar things?
We plan a student project for building kind of GUI for visualizing stripings 
and mappings, so I would try to avoid reinventing the wheel.

Depending on how complex your tool is, it may be better to use 
llapi_layout_from_file() or llapi_layout_from_xattr() to parse the binary 
layout directly from the file, rather than printing it out and then parsing the 
text again in userspace.

Cheers, Andreas

Am 21.01.2023 um 17:08 schrieb Andreas Dilger:
Hi Anna,
Beyond the number and size of OSTs and MDTs there isn't much information about 
the underlying storage available on the client.

The "lfs df -v" command will print a "f" at the end for flash (non-rotational) 
devices, if the storage is properly configured.  The "osc*.imports " parameter 
file will contain some information about the grant_block_size that can be used 
to distinguish ldiskfs (4096) vs. zfs backends (131072 or 1048576).

The size of the disks can often be inferred from 1/8 of the total OST size for 
standard 8+2 RAID configs, but this may vary and no actual device-level metrics 
are available on the client.

Even on the server, Lustre itself doesn't know or care much about the 
underlying storage devices beyond (non-)rotational state, so we don't track any 
of that.

Cheers, Andreas

On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss 
 wrote:

 Hi,

is it possible for a user (no root, so ssh to server) to find out the 
configuration of an OST?
How many devices are there in one OST 'pool' (for both ldiskfs and ZFS) and 
even which type of devices they are (nvme, ssd, hdd)? Maybe even speeds and 
raid-levels?

Additionally, how can a user find out the mapping of all available OSTs to OSSs 
easily?

Thanks
Anna

--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs

Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud







___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-l

Re: [lustre-discuss] User find out OST configuration

2023-02-09 Thread Anna Fuchs via lustre-discuss 

Thank you!

Depending on how complex your tool is, it may be better to use 
llapi_layout_from_file() or llapi_layout_from_xattr() to parse the 
binary layout directly from the file, rather than printing it out and 
then parsing the text again in userspace.
at first the tool should be runable by the user in userspace and help 
the user to understand the system/his data distribution without being 
dependent on admins, installed tools or versions.

The advanced one might include further features requiring root.

Best regards
Anna

On 1/24/23 03:13, Andreas Dilger wrote:

On Jan 23, 2023, at 10:01, Anna Fuchs  wrote:


Thanks!

Is it planned to introduce some metric propagation to the user?
For advanced users who are benchmarking stuff on remote systems it 
remains unclear which performance to expect if they can not access 
underlaying hardware metrics.  Sure, they can ask the admin to share 
the config, but it might be more convenient to be able to look it up, 
maybe.


Yes,  there is a longstanding open ticket to export some server 
statistics to the clients - https://jira.whamcloud.com/browse/LU-7880 
"add performance statistics to obd_statfs".  As the summary describes, 
this would export basic performance stats for each OST/MDT device to 
the client for current/peak x read/write x IOPS/bandwidth.  There are 
a number of potential uses for this, such as client/MDS selection of 
OSTs based on bandwidth/IOPS (beyond just "rotational" or 
"non-rotational"), userspace applications/libraries using it to 
determine if the OSTs are less busy (e.g. when to checkpoint), etc.


There aren't any plans to be able to export the storage "config" (e.g. 
RAID geometry) via Lustre since this is often opaque even on the 
server, and doesn't have any use on the client.  There was a 
discussion in the context of IO500 to write a script for collecting 
storage system configuration metadata for Lustre and other filesystems 
(e.g. list of OST/MDT devices, list of SCSI devices, PCI devices, CPU, 
RAM, network interfaces, etc.)


Additionally: if I try to find out the stripe location (lfs 
getstripe) and map this information to OST-specs (lctl get_param 
osc.*.*ost_conn_uuid), to find out how many different servers and 
networks are involved, the obdidx seems to be in dec-format, but the 
OST index in connections list is hex, which is not always obvious. 
 Is there a way to display it both in dec or both in hex?


There isn't currently an option for "lfs getstripe" to print in hex, 
but it would be possible to add a "--hex" option to print the fields 
in hex.  I've filed https://jira.whamcloud.com/browse/LU-16503 for 
tracking this issue.  I don't think it would be terribly complex to 
implement, but I also don't know if anyone is available to do this 
work right now.



Are there generally any tools for doing similar things?
We plan a student project for building kind of GUI for visualizing 
stripings and mappings, so I would try to avoid reinventing the wheel.



Cheers, Andreas


Am 21.01.2023 um 17:08 schrieb Andreas Dilger:

Hi Anna,
Beyond the number and size of OSTs and MDTs there isn't much 
information about the underlying storage available on the client.


The "lfs df -v" command will print a "f" at the end for flash 
(non-rotational) devices, if the storage is properly configured. 
 The "osc*.imports " parameter file will contain some information 
about the grant_block_size that can be used to distinguish ldiskfs 
(4096) vs. zfs backends (131072 or 1048576).


The size of the disks can often be inferred from 1/8 of the total 
OST size for standard 8+2 RAID configs, but this may vary and no 
actual device-level metrics are available on the client.


Even on the server, Lustre itself doesn't know or care much about 
the underlying storage devices beyond (non-)rotational state, so we 
don't track any of that.


Cheers, Andreas

On Jan 21, 2023, at 01:16, Anna Fuchs via lustre-discuss 
 wrote:


 Hi,

is it possible for a user (no root, so ssh to server) to find out 
the configuration of an OST?
How many devices are there in one OST 'pool' (for both ldiskfs and 
ZFS) and even which type of devices they are (nvme, ssd, hdd)? 
Maybe even speeds and raid-levels?


Additionally, how can a user find out the mapping of all available 
OSTs to OSSs easily?


Thanks
Anna
--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs
___
lustre-discuss mailing list
lustre-discuss@lists.lustre.org
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org


--
Anna Fuchs
Universität Hamburg
https://wr.informatik.uni-hamburg.de

anna.fu...@informatik.uni-hamburg.de
https://wr.informatik.uni-hamburg.de/people/anna_fuchs


Cheers, Andreas
--
Andreas Dilger
Lustre Principal Architect
Whamcloud






___
lustre-discuss mailing list
lustre-discuss@l