After the reboot the osd-disk changed from sdb to sda. This is a known
problem of linux (ubuntu)
root@bd-a:/etc/ceph# ceph-deploy -v osd activate bd-1:/dev/sda1:/dev/sda2
[ceph_deploy.conf][DEBUG ] found configuration file at:
/root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.3): /usr/bin/ceph-deploy -v osd
activate bd-1:/dev/sda1:/dev/sda2
[ceph_deploy.osd][DEBUG ] Activating cluster ceph disks
bd-1:/dev/sda1:/dev/sda2
[bd-1][DEBUG ] connected to host: bd-1
[bd-1][DEBUG ] detect platform information from remote host
[bd-1][DEBUG ] detect machine type
[ceph_deploy.osd][INFO ] Distro info: Ubuntu 14.04 trusty
[ceph_deploy.osd][DEBUG ] activating host bd-1 disk /dev/sda1
[ceph_deploy.osd][DEBUG ] will use init type: upstart
[bd-1][INFO ] Running command: ceph-disk-activate --mark-init upstart
--mount /dev/sda1
[bd-1][WARNIN] got monmap epoch 1
[bd-1][WARNIN] HDIO_DRIVE_CMD(identify) failed: Invalid argument
[bd-1][WARNIN] 2014-06-10 11:45:07.222697 7f5c111af800 -1 journal check:
ondisk fsid c8ce6ee2-f21b-4ba3-a20e-649224244b9a doesn't match expected
fcaaf66f-b7b7-4702-83a4-54832b7131fa, invalid (someone else's?) journal
[bd-1][WARNIN] HDIO_DRIVE_CMD(identify) failed: Invalid argument
[bd-1][WARNIN] HDIO_DRIVE_CMD(identify) failed: Invalid argument
[bd-1][WARNIN] HDIO_DRIVE_CMD(identify) failed: Invalid argument
[bd-1][WARNIN] 2014-06-10 11:45:08.125384 7f5c111af800 -1
filestore(/var/lib/ceph/tmp/mnt.LryOxo) could not find
23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory
[bd-1][WARNIN] 2014-06-10 11:45:08.320327 7f5c111af800 -1 created object
store /var/lib/ceph/tmp/mnt.LryOxo journal
/var/lib/ceph/tmp/mnt.LryOxo/journal for osd.4 fsid
08066b4a-3f36-4e3f-bd1e-15c006a09057
[bd-1][WARNIN] 2014-06-10 11:45:08.320367 7f5c111af800 -1 auth: error
reading file: /var/lib/ceph/tmp/mnt.LryOxo/keyring: can't open
/var/lib/ceph/tmp/mnt.LryOxo/keyring: (2) No such file or directory
[bd-1][WARNIN] 2014-06-10 11:45:08.320419 7f5c111af800 -1 created new key in
keyring /var/lib/ceph/tmp/mnt.LryOxo/keyring
[bd-1][WARNIN] added key for osd.4
[bd-1][INFO ] checking OSD status...
[bd-1][INFO ] Running command: ceph --cluster=ceph osd stat --format=json
[bd-1][WARNIN] there are 2 OSDs down
[bd-1][WARNIN] there are 2 OSDs out
root@bd-a:/etc/ceph# ceph -s
cluster 08066b4a-3f36-4e3f-bd1e-15c006a09057
health HEALTH_WARN 679 pgs degraded; 992 pgs stuck unclean; recovery
19/60 objects degraded (31.667%); clock skew detected on mon.bd-1
monmap e1: 3 mons at
{bd-0=xxx.xxx.xxx.20:6789/0,bd-1=xxx.xxx.xxx.21:6789/0,bd-2=xxx.xxx.xxx.22:6789/0},
election epoch 4034, quorum 0,1,2 bd-0,bd-1,bd-2
mdsmap e2815: 1/1/1 up {0=bd-2=up:active}, 2 up:standby
osdmap e1717: 6 osds: 4 up, 4 in
pgmap v46008: 992 pgs, 11 pools, 544 kB data, 20 objects
10324 MB used, 125 TB / 125 TB avail
19/60 objects degraded (31.667%)
2 active
679 active+degraded
311 active+remapped
root@bd-a:/etc/ceph# ceph osd tree
# id weight type name up/down reweight
-1 189.1 root default
-2 63.63 host bd-0
0 43.64 osd.0 up 1
3 19.99 osd.3 up 1
-3 63.63 host bd-1
1 43.64 osd.1 down 0
4 19.99 osd.4 down 0
-4 61.81 host bd-2
2 43.64 osd.2 up 1
5 18.17 osd.5 up 1
At this time i rebooted bd-1 once more and the osd-disk now was /dev/sdb.
So i tried once more to activate the osd:
root@bd-a:/etc/ceph# ceph-deploy -v osd activate bd-1:/dev/sdb1:/dev/sdb2
[ceph_deploy.conf][DEBUG ] found configuration file at:
/root/.cephdeploy.conf
[ceph_deploy.cli][INFO ] Invoked (1.5.3): /usr/bin/ceph-deploy -v osd
activate bd-1:/dev/sdb1:/dev/sdb2
[ceph_deploy.osd][DEBUG ] Activating cluster ceph disks
bd-1:/dev/sdb1:/dev/sdb2
[bd-1][DEBUG ] connected to host: bd-1
[bd-1][DEBUG ] detect platform information from remote host
[bd-1][DEBUG ] detect machine type
[ceph_deploy.osd][INFO ] Distro info: Ubuntu 14.04 trusty
[ceph_deploy.osd][DEBUG ] activating host bd-1 disk /dev/sdb1
[ceph_deploy.osd][DEBUG ] will use init type: upstart
[bd-1][INFO ] Running command: ceph-disk-activate --mark-init upstart
--mount /dev/sdb1
[bd-1][INFO ] checking OSD status...
[bd-1][INFO ] Running command: ceph --cluster=ceph osd stat --format=json
[bd-1][WARNIN] there are 2 OSDs down
[bd-1][WARNIN] there are 2 OSDs out
root@bd-a:/etc/ceph# ceph osd tree
# id weight type name up/down reweight
-1 189.1 root default
-2 63.63 host bd-0
0 43.64 osd.0 up 1
3 19.99 osd.3 up 1
-3 63.63 host bd-1
1 43.64 osd.1 down 0
4 19.99 osd.4 down 0
-4 61.81 host bd-2
2 43.64 osd.2 up 1
5 18.17 osd.5 up 1
root@bd-a:/etc/ceph# ceph -s
cluster 08066b4a-3f36-4e3f-bd1e-15c006a09057
health HEALTH_WARN 679 pgs degraded; 992 pgs stuck unclean; recovery
10/60 objects degraded (16.667%); clock skew detected on mon.bd-1
monmap e1: 3 mons at
{bd-0=xxx.xxx.xxx.20:6789/0,bd-1=xxx.xxx.xxx.21:6789/0,bd-2=xxx.xxx.xxx.22:6789/0},
election epoch 4060, quorum 0,1,2 bd-0,bd-1,bd-2
mdsmap e2823: 1/1/1 up {0=bd-2=up:active}, 2 up:standby
osdmap e1759: 6 osds: 4 up, 4 in
pgmap v46110: 992 pgs, 11 pools, 544 kB data, 20 objects
10320 MB used, 125 TB / 125 TB avail
10/60 objects degraded (16.667%)
679 active+degraded
313 active+remapped
root@bd-a:/etc/ceph#
After another reboot all was ok:
ceph -s
cluster 08066b4a-3f36-4e3f-bd1e-15c006a09057
health HEALTH_OK
monmap e1: 3 mons at
{bd-0=xxx.xxx.xxx.20:6789/0,bd-1=xxx.xxx.xxx.21:6789/0,bd-2=xxx.xxx.xxx.22:6789/0},
election epoch 4220, quorum 0,1,2 bd-0,bd-1,bd-2
mdsmap e2895: 1/1/1 up {0=bd-2=up:active}, 2 up:standby
osdmap e1939: 6 osds: 6 up, 6 in
pgmap v47099: 992 pgs, 11 pools, 551 kB data, 20 objects
117 MB used, 189 TB / 189 TB avail
992 active+clean
root@bd-a:~#
Is it possible for the author of ceph-deploy, to make the reboot needlessly
during these 2 steps ?
Then it would also be possible to use create instead of prepare+activate
Thank you,
Markus
--
MfG,
Markus Goldberg
--------------------------------------------------------------------------
Markus Goldberg Universität Hildesheim
Rechenzentrum
Tel +49 5121 88392822 Marienburger Platz 22, D-31141 Hildesheim, Germany
Fax +49 5121 88392823 email goldb...@uni-hildesheim.de
--------------------------------------------------------------------------
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com