Hello, I'm trying to deploy a ceph filestore cluster with LVM using ceph-ansible playbook. I've been fixing a couple of code blocks in ceph-ansible and ceph-disk/main.py and made some progress but now I'm stuck again; 'ceph-disk activate osd' fails.
Please let me just show you the error message and the output of 'ls': ~ # ceph-disk -v activate /dev/mapper/vg--hdd1-lv--hdd1p1 main_activate: path = /dev/mapper/vg--hdd1-lv--hdd1p1 get_dm_uuid: get_dm_uuid /dev/mapper/vg--hdd1-lv--hdd1p1 uuid path is /sys/dev/block/252:12/dm/uuid get_dm_uuid: get_dm_uuid /dev/mapper/vg--hdd1-lv--hdd1p1 uuid is part1-LVM-ETn7wXOmnesc9MNpleTYzP29jjOkp19J12ELrQez43LFPfdFc1dItn8EFF299401 command: Running command: /sbin/blkid -p -s TYPE -o value -- /dev/mapper/vg--hdd1-lv--hdd1p1 command: Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs mount: Mounting /dev/mapper/vg--hdd1-lv--hdd1p1 on /var/lib/ceph/tmp/mnt.cJDc7I with options noatime,largeio,inode64,swalloc command_check_call: Running command: /bin/mount -t xfs -o noatime,largeio,inode64,swalloc -- /dev/mapper/vg--hdd1-lv--hdd1p1 /var/lib/ceph/tmp/mnt.cJDc7I activate: Cluster uuid is 0bc0ea6d-ed8a-4ef0-9e82-ba6454a7214e command: Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid activate: Cluster name is ceph activate: OSD uuid is 5097be3f-349e-480d-8b0d-d68c13ae2f72 activate: OSD id is 1 activate: Initializing OSD... command_check_call: Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring mon getmap -o /var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap got monmap epoch 2 command: Running command: /usr/bin/timeout 300 ceph-osd --cluster ceph --mkfs --mkkey -i 1 --monmap /var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap --osd-data /var/lib/ceph/tmp/mnt.cJDc7I --osd-journal /var/lib/ceph/tmp/mnt.cJDc7I/journal --osd-uuid 5097be3f-349e-480d-8b0d-d68c13ae2f72 --keyring /var/lib/ceph/tmp/mnt.cJDc7I/keyring --setuser ceph --setgroup ceph mount_activate: Failed to activate unmount: Unmounting /var/lib/ceph/tmp/mnt.cJDc7I command_check_call: Running command: /bin/umount -- /var/lib/ceph/tmp/mnt.cJDc7I Traceback (most recent call last): File "/usr/sbin/ceph-disk", line 9, in <module> load_entry_point('ceph-disk==1.0.0', 'console_scripts', 'ceph-disk')() File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5251, in run main(sys.argv[1:]) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 5202, in main args.func(args) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3553, in main_activate reactivate=args.reactivate, File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3310, in mount_activate (osd_id, cluster) = activate(path, activate_key_template, init) File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 3486, in activate keyring=keyring, File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 2948, in mkfs '--setgroup', get_ceph_group(), File "/usr/lib/python2.7/dist-packages/ceph_disk/main.py", line 2895, in ceph_osd_mkfs raise Error('%s failed : %s' % (str(arguments), error)) ceph_disk.main.Error: Error: ['ceph-osd', '--cluster', 'ceph', '--mkfs', '--mkkey', '-i', u'1', '--monmap', '/var/lib/ceph/tmp/mnt.cJDc7I/activate.monmap', '--osd-data', '/var/lib/ceph/tmp/mnt.cJDc7I', '--osd-journal', '/var/lib/ceph/tmp/mnt.cJDc7I/journal', '--osd-uuid', u'5097be3f-349e-480d-8b0d-d68c13ae2f72', '--keyring', '/var/lib/ceph/tmp/mnt.cJDc7I/keyring', '--setuser', 'ceph', '--setgroup', 'ceph'] failed : 2017-03-14 16:01:10.051537 7fdc9a025a40 -1 filestore(/var/lib/ceph/tmp/mnt.cJDc7I) mkjournal error creating journal on /var/lib/ceph/tmp/mnt.cJDc7I/journal: (13) Permission denied 2017-03-14 16:01:10.051565 7fdc9a025a40 -1 OSD::mkfs: ObjectStore::mkfs failed with error -13 2017-03-14 16:01:10.051624 7fdc9a025a40 -1 ** ERROR: error creating empty object store in /var/lib/ceph/tmp/mnt.cJDc7I: (13) Permission denied ~ # ls -al /var/lib/ceph/tmp total 8 drwxr-xr-x 2 ceph ceph 4096 Mar 14 16:01 . drwxr-xr-x 11 ceph ceph 4096 Mar 14 11:12 .. -rwxr-xr-x 1 root root 0 Mar 14 11:12 ceph-disk.activate.lock -rwxr-xr-x 1 root root 0 Mar 14 11:44 ceph-disk.prepare.lock ~ # ls -l /dev/mapper/vg-*-lv-*p* lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd1-lv--hdd1p1 -> ../dm-12 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd2-lv--hdd2p1 -> ../dm-14 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd3-lv--hdd3p1 -> ../dm-16 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd4-lv--hdd4p1 -> ../dm-18 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd5-lv--hdd5p1 -> ../dm-20 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd6-lv--hdd6p1 -> ../dm-22 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd7-lv--hdd7p1 -> ../dm-24 lrwxrwxrwx 1 root root 8 Mar 14 13:46 /dev/mapper/vg--hdd8-lv--hdd8p1 -> ../dm-26 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--hdd9-lv--hdd9p1 -> ../dm-28 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p1 -> ../dm-11 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p2 -> ../dm-15 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p3 -> ../dm-19 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p4 -> ../dm-23 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd1-lv--ssd1p5 -> ../dm-27 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p1 -> ../dm-13 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p2 -> ../dm-17 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p3 -> ../dm-21 lrwxrwxrwx 1 root root 8 Mar 14 13:47 /dev/mapper/vg--ssd2-lv--ssd2p4 -> ../dm-25 ~ # ls -l /dev/dm-* brw-rw---- 1 root disk 252, 0 Mar 14 13:46 /dev/dm-0 brw-rw---- 1 root disk 252, 1 Mar 14 13:46 /dev/dm-1 brw-rw---- 1 root disk 252, 10 Mar 14 13:47 /dev/dm-10 brw-rw---- 1 ceph ceph 252, 11 Mar 14 13:47 /dev/dm-11 brw-rw---- 1 ceph ceph 252, 12 Mar 14 13:46 /dev/dm-12 brw-rw---- 1 ceph ceph 252, 13 Mar 14 13:47 /dev/dm-13 brw-rw---- 1 ceph ceph 252, 14 Mar 14 13:46 /dev/dm-14 brw-rw---- 1 ceph ceph 252, 15 Mar 14 13:47 /dev/dm-15 brw-rw---- 1 ceph ceph 252, 16 Mar 14 13:46 /dev/dm-16 brw-rw---- 1 ceph ceph 252, 17 Mar 14 13:47 /dev/dm-17 brw-rw---- 1 ceph ceph 252, 18 Mar 14 13:46 /dev/dm-18 brw-rw---- 1 ceph ceph 252, 19 Mar 14 13:47 /dev/dm-19 brw-rw---- 1 root disk 252, 2 Mar 14 13:46 /dev/dm-2 brw-rw---- 1 ceph ceph 252, 20 Mar 14 13:46 /dev/dm-20 brw-rw---- 1 ceph ceph 252, 21 Mar 14 13:47 /dev/dm-21 brw-rw---- 1 ceph ceph 252, 22 Mar 14 13:46 /dev/dm-22 brw-rw---- 1 ceph ceph 252, 23 Mar 14 13:47 /dev/dm-23 brw-rw---- 1 ceph ceph 252, 24 Mar 14 13:46 /dev/dm-24 brw-rw---- 1 ceph ceph 252, 25 Mar 14 13:47 /dev/dm-25 brw-rw---- 1 ceph ceph 252, 26 Mar 14 13:46 /dev/dm-26 brw-rw---- 1 ceph ceph 252, 27 Mar 14 13:47 /dev/dm-27 brw-rw---- 1 ceph ceph 252, 28 Mar 14 13:47 /dev/dm-28 brw-rw---- 1 root disk 252, 3 Mar 14 13:46 /dev/dm-3 brw-rw---- 1 root disk 252, 4 Mar 14 13:46 /dev/dm-4 brw-rw---- 1 root disk 252, 5 Mar 14 13:46 /dev/dm-5 brw-rw---- 1 root disk 252, 6 Mar 14 13:47 /dev/dm-6 brw-rw---- 1 root disk 252, 7 Mar 14 13:46 /dev/dm-7 brw-rw---- 1 root disk 252, 8 Mar 14 13:46 /dev/dm-8 brw-rw---- 1 root disk 252, 9 Mar 14 13:47 /dev/dm-9 Best Regards, Nicholas. -- You can find my PGP public key here: https://google.com/+DewrKim/about
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com