I already ran the "ceph-volume lvm activate --all " command right after I prepared (using "lvm prepare") those OSDs. Do I need to run the "activate" command again?
On Mon, Nov 5, 2018 at 1:24 PM, Alfredo Deza <ad...@redhat.com> wrote: > On Mon, Nov 5, 2018 at 12:54 PM Hayashida, Mami <mami.hayash...@uky.edu> > wrote: > > > > I commented out those lines and, yes, I was able to restart the system > and all the Filestore OSDs are now running. But when I cannot start > converted Bluestore OSDs (service). When I look up the log for osd.60, > this is what I see: > > Something/someone may have added those entries in fstab. It is > certainly not something that ceph-disk or ceph-volume would ever do. > Glad you found them and removed them. > > > > > > 2018-11-05 12:47:00.756794 7f1f2775ae00 0 set uid:gid to 64045:64045 > (ceph:ceph) > > 2018-11-05 12:47:00.756821 7f1f2775ae00 0 ceph version 12.2.9 ( > 9e300932ef8a8916fb3fda78c58691a6ab0f4217) luminous (stable), process > ceph-osd, pid 33706 > > 2018-11-05 12:47:00.776554 7f1f2775ae00 0 pidfile_write: ignore empty > --pid-file > > 2018-11-05 12:47:00.788716 7f1f2775ae00 0 load: jerasure load: lrc > load: isa > > 2018-11-05 12:47:00.788803 7f1f2775ae00 1 bdev create path > /var/lib/ceph/osd/ceph-60/block type kernel > > 2018-11-05 12:47:00.788818 7f1f2775ae00 1 bdev(0x564350f4ad80 > /var/lib/ceph/osd/ceph-60/block) open path /var/lib/ceph/osd/ceph-60/block > > 2018-11-05 12:47:00.789179 7f1f2775ae00 1 bdev(0x564350f4ad80 > /var/lib/ceph/osd/ceph-60/block) open size 10737418240 (0x280000000, > 10GiB) block_size 4096 (4KiB) rotational > > 2018-11-05 12:47:00.789257 7f1f2775ae00 1 > > bluestore(/var/lib/ceph/osd/ceph-60) > _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 > > 2018-11-05 12:47:00.789286 7f1f2775ae00 1 bdev(0x564350f4ad80 > /var/lib/ceph/osd/ceph-60/block) close > > 2018-11-05 12:47:01.075002 7f1f2775ae00 1 > > bluestore(/var/lib/ceph/osd/ceph-60) > _mount path /var/lib/ceph/osd/ceph-60 > > 2018-11-05 12:47:01.075069 7f1f2775ae00 1 bdev create path > /var/lib/ceph/osd/ceph-60/block type kernel > > 2018-11-05 12:47:01.075078 7f1f2775ae00 1 bdev(0x564350f4afc0 > /var/lib/ceph/osd/ceph-60/block) open path /var/lib/ceph/osd/ceph-60/block > > 2018-11-05 12:47:01.075391 7f1f2775ae00 1 bdev(0x564350f4afc0 > /var/lib/ceph/osd/ceph-60/block) open size 10737418240 (0x280000000, > 10GiB) block_size 4096 (4KiB) rotational > > 2018-11-05 12:47:01.075450 7f1f2775ae00 1 > > bluestore(/var/lib/ceph/osd/ceph-60) > _set_cache_sizes cache_size 1073741824 meta 0.4 kv 0.4 data 0.2 > > 2018-11-05 12:47:01.075536 7f1f2775ae00 1 bdev create path > /var/lib/ceph/osd/ceph-60/block.db type kernel > > 2018-11-05 12:47:01.075544 7f1f2775ae00 1 bdev(0x564350f4b200 > /var/lib/ceph/osd/ceph-60/block.db) open path /var/lib/ceph/osd/ceph-60/ > block.db > > 2018-11-05 12:47:01.075555 7f1f2775ae00 -1 bdev(0x564350f4b200 > /var/lib/ceph/osd/ceph-60/block.db) open open got: (13) Permission denied > > 2018-11-05 12:47:01.075573 7f1f2775ae00 -1 > > bluestore(/var/lib/ceph/osd/ceph-60) > _open_db add block device(/var/lib/ceph/osd/ceph-60/block.db) returned: > (13) Permission denied > > 2018-11-05 12:47:01.075589 7f1f2775ae00 1 bdev(0x564350f4afc0 > /var/lib/ceph/osd/ceph-60/block) close > > 2018-11-05 12:47:01.346356 7f1f2775ae00 -1 osd.60 0 OSD:init: unable to > mount object store > > 2018-11-05 12:47:01.346378 7f1f2775ae00 -1 ** ERROR: osd init failed: > (13) Permission denied > > If this has already been converted and it is ceph-volume trying to > start it, could you please try activating that one OSD with: > > ceph-volume lvm activate <OSD ID> <OSD FSID> > > And paste the output here, and then if possible, capture the relevant > part of that CLI call from /var/log/ceph/ceph-volume.log > > > > > > > > > > On Mon, Nov 5, 2018 at 12:34 PM, Hector Martin <hec...@marcansoft.com> > wrote: > >> > >> On 11/6/18 2:01 AM, Hayashida, Mami wrote: > >> > I did find in /etc/fstab entries like this for those 10 disks > >> > > >> > /dev/sdh1 /var/lib/ceph/osd/ceph-60 xfs noatime,nodiratime 0 0 > >> > > >> > Should I comment all 10 of them out (for osd.{60-69}) and try > rebooting > >> > again? > >> > >> Yes. Anything that references any of the old partitions that don't exist > >> (/dev/sdh1 etc) should be removed. The disks are now full-disk LVM PVs > >> and should have no partitions. > >> > >> -- > >> Hector Martin (hec...@marcansoft.com) > >> Public Key: https://mrcn.st/pub > > > > > > > > > > -- > > Mami Hayashida > > Research Computing Associate > > > > Research Computing Infrastructure > > University of Kentucky Information Technology Services > > 301 Rose Street | 102 James F. Hardymon Building > > Lexington, KY 40506-0495 > > mami.hayash...@uky.edu > > (859)323-7521 > -- *Mami Hayashida* *Research Computing Associate* Research Computing Infrastructure University of Kentucky Information Technology Services 301 Rose Street | 102 James F. Hardymon Building Lexington, KY 40506-0495 mami.hayash...@uky.edu (859)323-7521
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com