ceph osd destroy 70 --yes-i-really-mean-it I am guessing that’s a copy and paste mistake and should say 120.
Is the SSD @ /dev/sdh fully for the OSD120 is a partition on this SSD the journal and other partitions are for other SSD’s? On Wed, 7 Nov 2018 at 11:21 PM, Hayashida, Mami <mami.hayash...@uky.edu> wrote: > I would agree with that. So, here is what I am planning on doing today. > I will try this from scratch on a different OSD node from the very first > step and log input and output for every step. Here is the outline of what > I think (based on all the email exchanges so far) should happen. > > ******* > Trying to convert osd.120 to Bluestore. Data is on /sda/sdh. Filestore > Journal is on a partition drive (40GB) on /dev/sda. > > #Mark those OSDs out > ceph osd out 120 > > # Stop the OSDs > systemctl kill ceph-osd@120 > > # Unmount the filesystem > sudo umount /var/lib/ceph/osd/ceph-120 > > # Destroy the data > ceph-volume lvm zap /dev/sdh --destroy # data disk > ceph-volume lvm zap /dev/sda --destroy # ssd for wal and db > > # Inform the cluster > ceph osd destroy 70 --yes-i-really-mean-it > > # Check all the /etc/fstab and /etc/systemd/system to make sure that all > the references to the filesystem is gone. Run > ln -sf /dev/null /etc/systemd/system/ceph-disk@70.service > > # Create PVs, VGs, LVs > pvcreate /dev/sda # for wal and db > pvcreate /dev/sdh # for data > > vgcreate ssd0 /dev/sda > vgcreate hdd120 /dev/sdh > > lvcreate -L 40G -n db120 ssd0 > lvcreate -l 100%VG data120 hdd120 > > # Run ceph-volume > ceph-volume lvm prepare --bluestore --data hdd120/data120 --block.db > ssd0/db120 --osd-id 120 > > # Activate > ceph-volume lvm activate 120 <osd fsid> > > ****** > Does this sound right? > > On Tue, Nov 6, 2018 at 4:32 PM, Alfredo Deza <ad...@redhat.com> wrote: > >> It is pretty difficult to know what step you are missing if we are >> getting the `activate --all` command. >> >> Maybe if you try one by one, capturing each command, throughout the >> process, with output. In the filestore-to-bluestore guides we never >> advertise `activate --all` for example. >> >> Something is missing here, and I can't tell what it is. >> On Tue, Nov 6, 2018 at 4:13 PM Hayashida, Mami <mami.hayash...@uky.edu> >> wrote: >> > >> > This is becoming even more confusing. I got rid of those >> > ceph-disk@6[0-9].service >> (which had been symlinked to /dev/null). Moved >> /var/lib/ceph/osd/ceph-6[0-9] to /var/...../osd_old/. Then, I ran >> `ceph-volume lvm activate --all`. I got once again >> > >> > root@osd1:~# ceph-volume lvm activate --all >> > --> Activating OSD ID 67 FSID 17cd6755-76f9-4160-906c-1bf13d09fb3d >> > Running command: mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-67 >> > --> Absolute path not found for executable: restorecon >> > --> Ensure $PATH environment variable contains common executable >> locations >> > Running command: ceph-bluestore-tool --cluster=ceph prime-osd-dir --dev >> /dev/hdd67/data67 --path /var/lib/ceph/osd/ceph-67 >> > stderr: failed to read label for /dev/hdd67/data67: (2) No such file >> or directory >> > --> RuntimeError: command returned non-zero exit status: 1 >> > >> > But when I ran `df` and `mount` ceph-67 is the only one that exists. >> (and in /var/lib/ceph/osd/) >> > >> > root@osd1:~# df -h | grep ceph-6 >> > tmpfs 126G 0 126G 0% /var/lib/ceph/osd/ceph-67 >> > >> > root@osd1:~# mount | grep ceph-6 >> > tmpfs on /var/lib/ceph/osd/ceph-67 type tmpfs (rw,relatime) >> > >> > root@osd1:~# ls /var/lib/ceph/osd/ | grep ceph-6 >> > ceph-67 >> > >> > But in I cannot restart any of these 10 daemons (`systemctl start >> ceph-osd@6[0-9]`). >> > >> > I am wondering if I should zap these 10 osds and start over although at >> this point I am afraid even zapping may not be a simple task.... >> > >> > >> > >> > On Tue, Nov 6, 2018 at 3:44 PM, Hector Martin <hec...@marcansoft.com> >> wrote: >> >> >> >> On 11/7/18 5:27 AM, Hayashida, Mami wrote: >> >> > 1. Stopped osd.60-69: no problem >> >> > 2. Skipped this and went to #3 to check first >> >> > 3. Here, `find /etc/systemd/system | grep ceph-volume` returned >> >> > nothing. I see in that directory >> >> > >> >> > /etc/systemd/system/ceph-disk@60.service # and 61 - 69. >> >> > >> >> > No ceph-volume entries. >> >> >> >> Get rid of those, they also shouldn't be there. Then `systemctl >> >> daemon-reload` and continue, see if you get into a good state. >> basically >> >> feel free to nuke anything in there related to OSD 60-69, since >> whatever >> >> is needed should be taken care of by the ceph-volume activation. >> >> >> >> >> >> -- >> >> Hector Martin (hec...@marcansoft.com) >> >> Public Key: https://mrcn.st/pub >> > >> > >> > >> > >> > -- >> > Mami Hayashida >> > Research Computing Associate >> > >> > Research Computing Infrastructure >> > University of Kentucky Information Technology Services >> > 301 Rose Street | 102 James F. Hardymon Building >> > Lexington, KY 40506-0495 >> > mami.hayash...@uky.edu >> > (859)323-7521 >> > > > > -- > *Mami Hayashida* > > *Research Computing Associate* > Research Computing Infrastructure > University of Kentucky Information Technology Services > 301 Rose Street | 102 James F. Hardymon Building > Lexington, KY 40506-0495 > mami.hayash...@uky.edu > (859)323-7521 > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
