Hi! Trying to replace an OSD on a Jewel cluster (filestore data on HDD + journal device on SSD). I've set noout and removed the flapping drive (read errors) and replaced it with a new one.
I've taken down the osd UUID to be able to prepare the new disk with the same osd.ID. The journal device is the same as the previous one (should I delete the partition and recreate it?) However, running ceph-disk prepare returns: # ceph-disk -v prepare --cluster-uuid c51a2683-55dc-4634-9d9d-f0fec9a6f389 --osd-uuid dc49691a-2950-4028-91ea-742ffc9ed63f --journal-dev --data-dev --fs-type xfs /dev/sdo /dev/sda8 command: Running command: /usr/bin/ceph-osd --check-allows-journal -i 0 --log-file $run_dir/$cluster-osd-check.log --cluster ceph --setuser ceph --setgroup ceph command: Running command: /usr/bin/ceph-osd --check-wants-journal -i 0 --log-file $run_dir/$cluster-osd-check.log --cluster ceph --setuser ceph --setgroup ceph command: Running command: /usr/bin/ceph-osd --check-needs-journal -i 0 --log-file $run_dir/$cluster-osd-check.log --cluster ceph --setuser ceph --setgroup ceph Traceback (most recent call last): File "/usr/sbin/ceph-disk", line 9, in <module> load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5371, in run main(sys.argv[1:]) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5322, in main args.func(args) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 1900, in main Prepare.factory(args).prepare() File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 1896, in factory return PrepareFilestore(args) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 1909, in __init__ self.journal = PrepareJournal(args) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 2221, in __init__ raise Error('journal specified but not allowed by osd backend') ceph_disk.main.Error: Error: journal specified but not allowed by osd backend I tried googling first of course. It COULD be that we have set setuser_match_path globally in ceph.conf (like this bug report: https://tracker.ceph.com/issues/19642) since the cluster was created as dumpling a long time ago. Best practice to fix it? Create [osd.X] configs and set setuser_match_path in there instead for the old OSDs? Should I do any other steps preceding this if I want to use the same osd UUID? I've only stopped ceph-osd@21, removed the physical disk, inserted new one and tried running prepare. Kind Regards, David
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com