mount cephfs failes to (I added 3 MDS) anybody any ideas how to debug this further?
I used ceph-deploy to create the cluster, the xfs filesystem on the OSD's is okay, I can copy remove and open files on that partition so I asume it's something inside of ceph??? TIA Bernhard P.S.: Version is ceph version 0.67.2-16-gd41cf86 (d41cf866ee028ef7b821a5c37b991e85cbf3637f) on uptodate raring ---------- root@nuke36[/1]:/etc/ceph # ceph -s 2013-08-30 15:03:18.454701 7f3b7cd18700 1 -- :/0 messenger.start 2013-08-30 15:03:18.455460 7f3b7cd18700 1 -- :/1003684 --> 192.168.242.92:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- ?+0 0x7f3b7800e8f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.456412 7f3b7c517700 1 -- 192.168.242.36:0/1003684 learned my addr 192.168.242.36:0/1003684 2013-08-30 15:03:18.458069 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 1 ==== mon_map v1 ==== 776+0+0 (3609201999 0 0) 0x7f3b6c000c30 con 0x7f3b7800e4e0 2013-08-30 15:03:18.458308 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 2 ==== auth_reply(proto 2 0 Success) v1 ==== 33+0+0 (345113272 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.458612 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- auth(proto 2 32 bytes epoch 0) v1 -- ?+0 0x7f3b60001af0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.459532 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 3 ==== auth_reply(proto 2 0 Success) v1 ==== 206+0+0 (1084599267 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.459816 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- ?+0 0x7f3b600020d0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.460739 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 4 ==== auth_reply(proto 2 0 Success) v1 ==== 393+0+0 (496062897 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.460844 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=0+}) v2 -- ?+0 0x7f3b7800ed80 con 0x7f3b7800e4e0 2013-08-30 15:03:18.461118 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=2+,osdmap=0}) v2 -- ?+0 0x7f3b780079f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.461138 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_subscribe({monmap=2+,osdmap=0}) v2 -- ?+0 0x7f3b7800fa10 con 0x7f3b7800e4e0 2013-08-30 15:03:18.461813 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 5 ==== mon_map v1 ==== 776+0+0 (3609201999 0 0) 0x7f3b6c0008f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.462016 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c001340 con 0x7f3b7800e4e0 2013-08-30 15:03:18.463931 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- ?+0 0x7f3b7800b0f0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.463966 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 7 ==== osd_map(34..34 src has 1..34) v3 ==== 2483+0+0 (453205619 0 0) 0x7f3b6c0008c0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.464694 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 8 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c0010e0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.464749 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 9 ==== osd_map(34..34 src has 1..34) v3 ==== 2483+0+0 (453205619 0 0) 0x7f3b6c002720 con 0x7f3b7800e4e0 2013-08-30 15:03:18.464765 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 10 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3156621930 0 0) 0x7f3b6c002b20 con 0x7f3b7800e4e0 2013-08-30 15:03:18.468276 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 11 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+24040 (1092875540 0 2922658865) 0x7f3b6c002720 con 0x7f3b7800e4e0 2013-08-30 15:03:18.510756 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 --> 192.168.242.92:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- ?+0 0x7f3b7800b0d0 con 0x7f3b7800e4e0 2013-08-30 15:03:18.512490 7f3b76ffd700 1 -- 192.168.242.36:0/1003684 <== mon.3 192.168.242.92:6789/0 12 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+497 (1155462804 0 3461792647) 0x7f3b6c001080 con 0x7f3b7800e4e0 cluster f57cdca3-7222-4095-853b-03727461f725 health HEALTH_OK monmap e1: 5 mons at {atom01=192.168.242.31:6789/0,atom02=192.168.242.32:6789/0,nuke36=192.168.242.36:6789/0,ping=192.168.242.92:6789/0,pong=192.168.242.93:6789/0}, election epoch 42, quorum 0,1,2,3,4 atom01,atom02,nuke36,ping,pong osdmap e34: 2 osds: 2 up, 2 in pgmap v367: 1192 pgs: 1192 active+clean; 9788 bytes data, 94460 KB used, 3722 GB / 3722 GB avail mdsmap e17: 1/1/1 up {0=atom02=up:active}, 2 up:standby 2013-08-30 15:03:18.516793 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 mark_down 0x7f3b7800e4e0 -- 0x7f3b7800e280 2013-08-30 15:03:18.517204 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 mark_down_all 2013-08-30 15:03:18.517921 7f3b7cd18700 1 -- 192.168.242.36:0/1003684 shutdown complete. root@nuke36[/1]:/etc/ceph # mount -t ceph 192.168.242.36:6789:/ /mnt/ceph/ mount error 5 = Input/output error ---------- Am 30.08.2013 11:48:35, schrieb Bernhard Glomm: > Hi all, > > due to a problem with ceph-deploy I currently use > > deb http://gitbuilder.ceph.com/ceph-deb-raring-x86_64-basic/ref/wip-4924/ > raring main > (ceph version 0.67.2-16-gd41cf86 (d41cf866ee028ef7b821a5c37b991e85cbf3637f)) > > Now the initialization of the cluster works like a charm, > ceph health is okay, > just the mapping of the created rbd is failing. > > --------------------- > root@ping[/1]:~ # ceph osd pool delete kvm-pool kvm-pool > --yes-i-really-really-mean-it > pool 'kvm-pool' deleted > root@ping[/1]:~ # ceph osd lspools > > 0 data,1 metadata,2 rbd, > root@ping[/1]:~ # > root@ping[/1]:~ # ceph osd pool create kvm-pool 1000 > pool 'kvm-pool' created > root@ping[/1]:~ # ceph osd lspools > 0 data,1 metadata,2 rbd,4 kvm-pool, > root@ping[/1]:~ # ceph osd pool set kvm-pool min_size 2 > set pool 4 min_size to 2 > root@ping[/1]:~ # ceph osd dump | grep 'rep size' > pool 0 'data' rep size 2 min_size 1 crush_ruleset 0 object_hash rjenkins pg_num 64 pgp_num 64 last_change 1 owner 0 crash_replay_interval 45 > pool 1 'metadata' rep size 2 min_size 1 crush_ruleset 1 object_hash rjenkins > pg_num 64 pgp_num 64 last_change 1 owner 0 > pool 2 'rbd' rep size 2 min_size 1 crush_ruleset 2 object_hash rjenkins > pg_num 64 pgp_num 64 last_change 1 owner 0 > pool 4 'kvm-pool' rep size 2 min_size 2 crush_ruleset 0 object_hash rjenkins > pg_num 1000 pgp_num 1000 last_change 33 owner 0 > root@ping[/1]:~ # rbd create atom03.cimg --size 4000 --pool kvm-pool > root@ping[/1]:~ # rbd create atom04.cimg --size 4000 --pool kvm-pool > root@ping[/1]:~ # rbd ls kvm-pool > atom03.cimg > atom04.cimg > root@ping[/1]:~ # rbd --image atom03.cimg --pool kvm-pool info > rbd image 'atom03.cimg': > size 4000 MB in 1000 objects > order 22 (4096 KB objects) > block_name_prefix: rb.0.114d.2ae8944a > format: 1 > root@ping[/1]:~ # rbd --image atom04.cimg --pool kvm-pool info > rbd image 'atom04.cimg': > size 4000 MB in 1000 objects > order 22 (4096 KB objects) > block_name_prefix: rb.0.127d.74b0dc51 > format: 1 > root@ping[/1]:~ # rbd map atom03.cimg --pool kvm-pool --id admin > rbd: '/sbin/udevadm settle' failed! (256) > root@ping[/1]:~ # rbd map --pool kvm-pool --image atom03.cimg --id admin > --keyring /etc/ceph/ceph.client.admin.keyring > ^Crbd: '/sbin/udevadm settle' failed! (2) > root@ping[/1]:~ # rbd map kvm-pool/atom03.cimg --id admin --keyring > /etc/ceph/ceph.client.admin.keyring > rbd: '/sbin/udevadm settle' failed! (256) > --------------------- > > Do I miss something? > But I think this set of commands worked perfectly with cuttlefish? > > TIA > > Bernhard > > -- > > > > > > > > > Bernhard Glomm > IT Administration > > > > Phone: > > +49 (30) 86880 134 > > Fax: > > +49 (30) 86880 100 > > Skype: > > bernhard.glomm.ecologic > > > > > > > > > > > Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany > GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464 > Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH > > > _______________________________________________ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Bernhard Glomm IT Administration Phone: +49 (30) 86880 134 Fax: +49 (30) 86880 100 Skype: bernhard.glomm.ecologic Ecologic Institut gemeinnützige GmbH | Pfalzburger Str. 43/44 | 10717 Berlin | Germany GF: R. Andreas Kraemer | AG: Charlottenburg HRB 57947 | USt/VAT-IdNr.: DE811963464 Ecologic™ is a Trade Mark (TM) of Ecologic Institut gemeinnützige GmbH
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com