Hi Rick,
Thank you, that pointed me i the right direction!
The MGD still had OST0003 as inactive:
[root@mds ~]# lctl --device MGS llog_print lustre-MDT0000
<snip>
- { index: 32, event: add_uuid, nid: 192.168.6.5@tcp(0x20000c0a80605), node:
192.168.6.5@tcp }
- { index: 33, event: attach, device: lustre-OST0003-osc-MDT0000, type: osc,
UUID: lustre-MDT0000-mdtlov_UUID }
- { index: 34, event: setup, device: lustre-OST0003-osc-MDT0000, UUID:
lustre-OST0003_UUID, node: 192.168.6.5@tcp }
- { index: 35, event: add_osc, device: lustre-MDT0000-mdtlov, ost:
lustre-OST0003_UUID, index: 3, gen: 1 }
- { index: 38, event: conf_param, device: lustre-OST0003-osc-MDT0000,
parameter: osc.active=0 }
After activting it with 'lctl conf_param lustre-OST0003.osc.active=1', the
client can see the OST again.
# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]
lustre-OST0003_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:3]
lustre-OST0004_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:4]
filesystem_summary: 486.2T 222.5T 263.7T 46% /lustre
Thank you all for the help! Lustre can get quite complicated when things do not
work as expected ;)
gr,
Martin Balvers
________________________________
From: Mohr, Rick <[email protected]>
Sent: Thursday, August 14, 2025 15:58
To: BALVERS Martin <[email protected]>; [email protected]
<[email protected]>
Subject: Re: [EXTERNAL] Re: [lustre-discuss] Mistake while adding OST - OST0003
: Invalid argument
WARNING - EXTERNAL SENDER - BE CYBERSAFE
Martin,
Have you tried doing a writeconf on everything to regenerate the config logs?
If the fs is not in use atm, this might be a good time to try it.
--Rick
On 8/14/25, 3:11 AM, "lustre-discuss on behalf of BALVERS Martin via
lustre-discuss" <[email protected]
<mailto:[email protected]> on behalf of
[email protected] <mailto:[email protected]>> wrote:
Hi Thomas,
Thanks for the reply.
Here is the output:
[root@mds ~]# lctl get_param osp.lustre-OST0003-osc-MDT0000.active
osp.lustre-OST0003-osc-MDT0000.active=1
The problem is caused by the fact that I replaced a previously added OST using
the --replace option in the mkfs.lustre command.
OST0004 was added correctly in one go, and that one is visible to the client,
so I don't think it is a client problem. The client can reach the OSS that has
OST0003.
It seems that the according to the MDS everything is fine. It is only the 'lfs
df' command that shows a problem. Regular 'df' shows all the space of the
lustre fs, but 'lfs df' does not.
# df -h
Filesystem Size Used Avail Use% Mounted on
192.168.6.1@tcp:/lustre 487T 223T 264T 46% /lustre
# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]
OST0003 : Invalid argument
lustre-OST0004_UUID 97.2T 7.0M 97.2T 1% /lustre[OST:4]
filesystem_summary: 388.9T 222.5T 166.4T 58% /lustre
The fs is not in use at the moment, no data has been written to it since adding
OST0003 an 4. If there is a way to fix this removing and adding OST0003 again
then I can try that. As long as the data that is on the other OST's is not lost.
Regards,
Martin Balvers
Ce message électronique et tous les fichiers attachés qu'il contient sont
confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils
sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à
son émetteur. Les idées et opinions présentées dans ce message sont celles de
son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une
quelconque de ses filiales. La publication, l'usage, la distribution,
l'impression ou la copie non autorisée de ce message et des attachements qu'il
contient sont strictement interdits.
This e-mail and any files transmitted with it are confidential and intended
solely for the use of the individual to whom it is addressed. If you have
received this email in error please send it back to the person that sent it to
you. Any views or opinions presented are solely those of its author and do not
necessarily represent those of DANONE or any of its subsidiary companies.
Unauthorized publication, use, dissemination, forwarding, printing or copying
of this email and its associated attachments is strictly prohibited.
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
