Hi,
I really hope someone can help me.
I setup a new lustre filesystem (version 2.15.1) and copied all my data from my
old filesystem over to the new one.
I then wanted to add two old server to the new filesystem. I create a new zpool
and created the lustre fs with:
mkfs.lustre --reformat --ost --backfstype=zfs --fsname=lustre --index=3
--mgsnode=192.168.6.1@tcp lustre/ost
Then I noticed the pool was way smaller than expected. I did this on the mgs"
lctl --device MGS conf_param lustre-OST0003.target.active=0
Destroyed the zpool on the oss, wiped the disks and recreated the pool. Then
used:
mkfs.lustre --reformat --ost --backfstype=zfs --fsname=lustre --index=3
--replace --mgsnode=192.168.6.1@tcp lustre/ost
lctl --device 13 activate
Now I see this from a client, OST003: invalid argument:
# lfs df -h
UUID bytes Used Available Use% Mounted on
lustre-MDT0000_UUID 11.4T 2.8T 8.6T 25% /lustre[MDT:0]
lustre-OST0000_UUID 97.2T 74.4T 22.8T 77% /lustre[OST:0]
lustre-OST0001_UUID 97.2T 74.0T 23.2T 77% /lustre[OST:1]
lustre-OST0002_UUID 97.2T 74.1T 23.1T 77% /lustre[OST:2]
OST0003 : Invalid argument
lustre-OST0004_UUID 97.2T 8.0M 97.2T 1% /lustre[OST:4]
filesystem_summary: 388.9T 222.5T 166.4T 58% /lustre
How do I recover from this?
extra info:
[root@mds ~]# lctl dl
0 UP osd-zfs MGS-osd MGS-osd_UUID 4
1 UP mgs MGS MGS 18
2 UP mgc MGC192.168.6.1@tcp 74a2f74d-7309-414a-9d07-b5cf97a990a4 4
3 UP osd-zfs lustre-MDT0000-osd lustre-MDT0000-osd_UUID 11
4 UP mds MDS MDS_uuid 2
5 UP lod lustre-MDT0000-mdtlov lustre-MDT0000-mdtlov_UUID 3
6 UP mdt lustre-MDT0000 lustre-MDT0000_UUID 20
7 UP mdd lustre-MDD0000 lustre-MDD0000_UUID 3
8 UP qmt lustre-QMT0000 lustre-QMT0000_UUID 3
9 UP lwp lustre-MDT0000-lwp-MDT0000 lustre-MDT0000-lwp-MDT0000_UUID 4
10 UP osp lustre-OST0000-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
11 UP osp lustre-OST0002-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
12 UP osp lustre-OST0001-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
13 UP osp lustre-OST0003-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
14 UP osp lustre-OST0004-osc-MDT0000 lustre-MDT0000-mdtlov_UUID 4
[root@mds ~]# lctl get_param lov.*-mdtlov.target_obd
lov.lustre-MDT0000-mdtlov.target_obd=
0: lustre-OST0000_UUID ACTIVE
1: lustre-OST0001_UUID ACTIVE
2: lustre-OST0002_UUID ACTIVE
3: lustre-OST0003_UUID ACTIVE
4: lustre-OST0004_UUID ACTIVE
[root@mds ~]# lctl get_param mgs.MGS.live.*
mgs.MGS.live.lustre=
fsname: lustre
flags: 0x20 gen: 25
lustre-MDT0000
lustre-OST0000
lustre-OST0001
lustre-OST0002
lustre-OST0003
lustre-OST0004
Secure RPC Config Rules:
imperative_recovery_state:
state: full
nonir_clients: 0
nidtbl_version: 9
notify_duration_total: 0.002020998
notify_duation_max: 0.000636370
notify_count: 7
mgs.MGS.live.params=
fsname: params
flags: 0x20 gen: 1
Secure RPC Config Rules:
imperative_recovery_state:
state: full
nonir_clients: 0
nidtbl_version: 2
notify_duration_total: 0.000000000
notify_duation_max: 0.000000000
notify_count: 0
Ce message électronique et tous les fichiers attachés qu'il contient sont
confidentiels et destinés exclusivement à l'usage de la personne à laquelle ils
sont adressés. Si vous avez reçu ce message par erreur, merci de le retourner à
son émetteur. Les idées et opinions présentées dans ce message sont celles de
son auteur, et ne représentent pas nécessairement celles de DANONE ou d'une
quelconque de ses filiales. La publication, l'usage, la distribution,
l'impression ou la copie non autorisée de ce message et des attachements qu'il
contient sont strictement interdits.
This e-mail and any files transmitted with it are confidential and intended
solely for the use of the individual to whom it is addressed. If you have
received this email in error please send it back to the person that sent it to
you. Any views or opinions presented are solely those of its author and do not
necessarily represent those of DANONE or any of its subsidiary companies.
Unauthorized publication, use, dissemination, forwarding, printing or copying
of this email and its associated attachments is strictly prohibited.
_______________________________________________
lustre-discuss mailing list
[email protected]
http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
