Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication
You have saved my day! Thank you so much :) no seems to be working not sure why it has that behaivor... Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Apr 14, 2015, at 12:34 AM, oyym...@gmail.commailto:oyym...@gmail.com oyym...@gmail.commailto:oyym...@gmail.com wrote: UUID strings in the file /etc/fstab refer to devices. you can retrieve UUID using command blkid, for instance: # blkid /dev/sda1 /dev/sda1: UUID=1bcb1cbb-abd2-4cfa-bf89-1726ea6cd2fa TYPE=xfs oyym...@gmail.commailto:oyym...@gmail.com From: Jesus Chavez (jeschave)mailto:jesch...@cisco.com Date: 2015-04-14 07:03 To: oyym...@gmail.commailto:oyym...@gmail.com CC: ceph-usersmailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication How did you get the UUIDs without mounting the osds? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Apr 10, 2015, at 11:47 PM, oyym...@gmail.commailto:oyym...@gmail.com oyym...@gmail.commailto:oyym...@gmail.com wrote: In my cluster, mount osd devices manually in /etc/fstab, for instances: vi /etc/fstab # ... UUID=98c87892-21cd-481b-9619-1a4a65620d79 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=d56aeb7b-6160-4872-91dc-346b39019081 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=db9782a6-b661-46b5-b993-a4d96bdc37d8 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=ec95aa19-127d-410b-86a3-d68ee6baff2e /var/lib/ceph/osd/ceph-3 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=44694045-d9a3-4df4-948c-a77696d22406 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 mount -a mount /dev/sda12 on /var/lib/ceph/osd/ceph-0 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sda13 on /var/lib/ceph/osd/ceph-1 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb2 on /var/lib/ceph/osd/ceph-3 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb3 on /var/lib/ceph/osd/ceph-4 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) reboot ceph osd tree # id weight type name up/down reweight -1 22.75 root default -2 4.55 host storage1 0 0.91 osd.0 up 1 1 0.91 osd.1 up 1 2 0.91 osd.2 up 1 3 0.91 osd.3 up 1 4 0.91 osd.4 up 1 -3 4.55 host storage2 5 0.91 osd.5 up 1 6 0.91 osd.6 up 1 7 0.91 osd.7 up 1 8 0.91 osd.8 up 1 9 0.91 osd.9 up 1 -4 4.55 host lkl-storage3 10 0.91 osd.10 up 1 11 0.91 osd.11 up 1 12 0.91 osd.12 up 1 13 0.91 osd.13 up 1 14 0.91 osd.14 up 1 -5 4.55 host storage4 15 0.91 osd.15 up 1 16 0.91 osd.16 up 1 17 0.91 osd.17 up 1 18 0.91 osd.18 up 1 19 0.91 osd.19 up 1 -6 4.55 host storage5 20 0.91 osd.20 up 1 21 0.91 osd.21 up 1 22 0.91 osd.22 up 1 23 0.91 osd.23 up 1 24 0.91 osd.24 up 1 oyym...@gmail.commailto:oyym...@gmail.com From: Jesus Chavez (jeschave)mailto:jesch...@cisco.com Date: 2015-04-10 22:17 To: oyym...@gmail.commailto:oyym...@gmail.com Subject: Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication Hi there, just to be sure if you could figure how to fix this problem? Thanks image005(04-11-12-39-14).png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006(04-11-12-39-14).gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 27, 2015, at 8:05 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Did you know how to solve it? it is really weird… Thanks! image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006.gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you
Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication
How did you get the UUIDs without mounting the osds? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Apr 10, 2015, at 11:47 PM, oyym...@gmail.commailto:oyym...@gmail.com oyym...@gmail.commailto:oyym...@gmail.com wrote: In my cluster, mount osd devices manually in /etc/fstab, for instances: vi /etc/fstab # ... UUID=98c87892-21cd-481b-9619-1a4a65620d79 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=d56aeb7b-6160-4872-91dc-346b39019081 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=db9782a6-b661-46b5-b993-a4d96bdc37d8 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=ec95aa19-127d-410b-86a3-d68ee6baff2e /var/lib/ceph/osd/ceph-3 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 UUID=44694045-d9a3-4df4-948c-a77696d22406 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,inode64,logbsize=256k,delaylog 0 3 mount -a mount /dev/sda12 on /var/lib/ceph/osd/ceph-0 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sda13 on /var/lib/ceph/osd/ceph-1 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb1 on /var/lib/ceph/osd/ceph-2 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb2 on /var/lib/ceph/osd/ceph-3 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) /dev/sdb3 on /var/lib/ceph/osd/ceph-4 type xfs (rw,noatime,seclabel,attr2,inode64,logbsize=256k,noquota) reboot ceph osd tree # id weight type name up/down reweight -1 22.75 root default -2 4.55 host storage1 0 0.91 osd.0 up 1 1 0.91 osd.1 up 1 2 0.91 osd.2 up 1 3 0.91 osd.3 up 1 4 0.91 osd.4 up 1 -3 4.55 host storage2 5 0.91 osd.5 up 1 6 0.91 osd.6 up 1 7 0.91 osd.7 up 1 8 0.91 osd.8 up 1 9 0.91 osd.9 up 1 -4 4.55 host lkl-storage3 10 0.91 osd.10 up 1 11 0.91 osd.11 up 1 12 0.91 osd.12 up 1 13 0.91 osd.13 up 1 14 0.91 osd.14 up 1 -5 4.55 host storage4 15 0.91 osd.15 up 1 16 0.91 osd.16 up 1 17 0.91 osd.17 up 1 18 0.91 osd.18 up 1 19 0.91 osd.19 up 1 -6 4.55 host storage5 20 0.91 osd.20 up 1 21 0.91 osd.21 up 1 22 0.91 osd.22 up 1 23 0.91 osd.23 up 1 24 0.91 osd.24 up 1 oyym...@gmail.commailto:oyym...@gmail.com From: Jesus Chavez (jeschave)mailto:jesch...@cisco.com Date: 2015-04-10 22:17 To: oyym...@gmail.commailto:oyym...@gmail.com Subject: Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication Hi there, just to be sure if you could figure how to fix this problem? Thanks image005(04-11-12-39-14).png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006(04-11-12-39-14).gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 27, 2015, at 8:05 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Did you know how to solve it? it is really weird… Thanks! image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006.gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 25, 2015, at 1:31 AM, oyym...@gmail.commailto:oyym...@gmail.com wrote: Hi，Jesus I encountered similar problem. 1. shut down one of nodes， but all osds can't reactive on the node after reboot. 2. run service ceph restart manually, got the same error message: [root@storage4 ~]# /etc/init.d/ceph start === osd.15 === 2015-03-23 14:43:32.399811 7fed0fcf4700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication 2015-03-23 14:43:32.399814 7fed0fcf4700 0 librados: osd.15 initialization error (2) No such file or directory Error connecting to cluster: ObjectNotFound failed: 'timeout 30 /usr/bin/ceph -c /etc/ceph/ceph.conf --name=osd.15 --keyring=/var/lib/ceph
[ceph-users] OSDs remain down
Hi all , still have a lot of problems when building power fail… There is only one simple node where OSDs remains down after reboot and there is something weird, 1 from 12 OSDs gets up after reboot but just one…Here is an example: Filesystem Size Used Avail Use% Mounted on /dev/mapper/rhel-root 50G 4.8G 46G 10% / devtmpfs 126G 0 126G 0% /dev tmpfs 126G 80K 126G 1% /dev/shm tmpfs 126G 9.5M 126G 1% /run tmpfs 126G 0 126G 0% /sys/fs/cgroup /dev/sda1 494M 165M 330M 34% /boot /dev/mapper/rhel-home 36G 7.1G 29G 20% /home /dev/sdf1 3.7T 380M 3.7T 1% /var/lib/ceph/osd/ceph-43 as you can see inly one OSD comes up. =( I don’t know what else to do is the only node that is affecting the production [root@capricornio ceph]# ceph osd tree # idweight type name up/down reweight -1 435.6 root default -2 43.56 host sagitario 0 3.63osd.0 up 1 12 3.63osd.12 up 1 22 3.63osd.22 up 1 32 3.63osd.32 up 1 42 3.63osd.42 up 1 52 3.63osd.52 up 1 62 3.63osd.62 up 1 72 3.63osd.72 up 1 82 3.63osd.82 up 1 92 3.63osd.92 up 1 102 3.63osd.102 up 1 112 3.63osd.112 up 1 -3 43.56 host capricornio 1 3.63osd.1 up 1 10 3.63osd.10 up 1 20 3.63osd.20 up 1 30 3.63osd.30 up 1 40 3.63osd.40 up 1 50 3.63osd.50 up 1 60 3.63osd.60 up 1 70 3.63osd.70 up 1 80 3.63osd.80 up 1 90 3.63osd.90 up 1 100 3.63osd.100 up 1 110 3.63osd.110 up 1 -4 43.56 host escorpion 2 3.63osd.2 up 1 11 3.63osd.11 up 1 21 3.63osd.21 up 1 31 3.63osd.31 up 1 41 3.63osd.41 up 1 51 3.63osd.51 up 1 61 3.63osd.61 up 1 71 3.63osd.71 up 1 81 3.63osd.81 up 1 91 3.63osd.91 up 1 101 3.63osd.101 up 1 111 3.63osd.111 up 1 -5 43.56 host tauro 3 3.63osd.3 down0 13 3.63osd.13 down0 23 3.63osd.23 down0 33 3.63osd.33 down1 43 3.63osd.43 up 1 53 3.63osd.53 down0 63 3.63osd.63 down0 73 3.63osd.73 down0 83 3.63osd.83 down0 93 3.63osd.93 down0 103 3.63osd.103 down0 113 3.63osd.113 down0 -6 43.56 host geminis 4 3.63osd.4 up 1 14 3.63osd.14 up 1 24 3.63osd.24 up 1 34 3.63osd.34 up 1 44 3.63osd.44 up 1 54 3.63osd.54 up 1 64 3.63osd.64 up 1 74 3.63osd.74 up 1 84 3.63osd.84 up 1 94 3.63osd.94 up 1 104 3.63osd.104 up 1 114 3.63osd.114 up 1 -7 43.56 host aries 5 3.63osd.5 up 1 15 3.63osd.15 up 1 25 3.63osd.25 up 1 35 3.63osd.35 up 1 45 3.63osd.45 up 1 55 3.63osd.55 up 1 65 3.63osd.65 up 1 75 3.63osd.75 up 1 85 3.63osd.85 up 1 95 3.63osd.95 up 1 105 3.63osd.105 up 1 115 3.63osd.115 up 1 -8 43.56 host cancer 6 3.63osd.6 up 1 16 3.63osd.16 up 1 26 3.63osd.26 up 1 36 3.63
Re: [ceph-users] ERROR: missing keyring, cannot use cephx for authentication
the thing is that the devices are not mounting after reboot… Any ideas? [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 23, 2015, at 3:37 PM, Thomas Foster thomas.foste...@gmail.commailto:thomas.foste...@gmail.com wrote: check your server where that osd is located and see if you have created the directory correctly. If you didn't create it correctly you would get that error message. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] ERROR: missing keyring, cannot use cephx for authentication
Hi all, I did HA failover test shutting down 1 node and I see that only 1 OSD came up after reboot: [root@geminis ceph]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/rhel-root 50G 4.5G 46G 9% / devtmpfs 126G 0 126G 0% /dev tmpfs 126G 80K 126G 1% /dev/shm tmpfs 126G 9.9M 126G 1% /run tmpfs 126G 0 126G 0% /sys/fs/cgroup /dev/sda1 494M 165M 330M 34% /boot /dev/mapper/rhel-home 36G 44M 36G 1% /home /dev/sdc1 3.7T 134M 3.7T 1% /var/lib/ceph/osd/ceph-14 If I run service ceph restart I got this error message… Stopping Ceph osd.94 on geminis...done === osd.94 === 2015-03-23 15:05:41.632505 7fe7b9941700 -1 monclient(hunting): ERROR: missing keyring, cannot use cephx for authentication 2015-03-23 15:05:41.632508 7fe7b9941700 0 librados: osd.94 initialization error (2) No such file or directory Error connecting to cluster: ObjectNotFound failed: 'timeout 30 /usr/bin/ceph -c /etc/ceph/ceph.conf --name=osd.94 --keyring=/var/lib/ceph/osd/ceph-94/keyring osd crush create-or-move -- 94 0.05 host=geminis root=default I have ceph.conf and ceph.client.admin.keyring under /etc/ceph: [root@geminis ceph]# ls /etc/ceph ceph.client.admin.keyring ceph.conf rbdmap tmp1OqNFi tmptQ0a1P [root@geminis ceph]# does anybody know what could be wrong? Thanks [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Forece Removal
Yeah Illdo that, the version is .87.1-0.el7 Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 5:02 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Yes, at this point, I'd export the CRUSH, edit it and import it back in. What version are you running? Robert LeBlanc Sent from a mobile device please excuse any typos. On Mar 20, 2015 4:28 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: thats what you sayd? [root@capricornio ~]# ceph auth del osd.9 entity osd.9 does not exist [root@capricornio ~]# ceph auth del osd.19 entity osd.19 does not exist image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006.gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 20, 2015, at 4:13 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Does it show DNE in the entry? That stands for Does Not Exist. It will disappear on it's own after a while. I don't know what the timeout is, but they have always gone away within 24 hours. I've edited the CRUSH map before and I don't think it removed it when it was already DNE, I just had to wait for it to go away on it's own. On Fri, Mar 20, 2015 at 3:55 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Maybe I should Edit the crushmap and delete osd... Is that a way yo force them? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 2:21 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Removing the OSD from the CRUSH map and deleting the auth key is how you force remove an OSD. The OSD can no longer participate in the cluster, even if it does come back to life. All clients forget about the OSD when the new CRUSH map is distributed. On Fri, Mar 20, 2015 at 11:19 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Any idea how to forcé remove ? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 Begin forwarded message: From: Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr Date: March 20, 2015 at 3:49:11 AM CST To: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com Cc: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] OSD Forece Removal Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks [cid:] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph
Re: [ceph-users] Question Blackout
Thank you Craig thats the Kind of answer I was expecting for :) actually we have had an outage and 1 server never came up but was because of human error and 1 osd from order node were down so we Couldnt even access to the filesystem but At last I saw that the 90% of the nodes didnt crush or something :) So thats fine for Now Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 4:39 PM, Craig Lewis cle...@centraldesktop.commailto:cle...@centraldesktop.com wrote: I'm not a CephFS user, but I have had a few cluster outages. Each OSD has a journal, and Ceph ensures that a write is in all of the journals (primary and replicas) before it acknowledges the write. If an OSD process crashes, it replays the journal on startup, and recovers the write. I've lost power at my data center, and had the whole cluster down. Ceph came back up when power was restored without me getting involved. You might want the paid support package. For extra piece of mind, you can get a paid cluster review, and an engineer will go through your use case with you. On Tue, Mar 17, 2015 at 8:32 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Hi everyone, I am ready to launch ceph on production but there is one thing that keeps on my mind... If there was a Blackout where all the ceph nodes went off what would really happen with the filesystem? It would get corrupt? Or ceph has any Kind of mechanism to survive to something like that? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] OSD remains down
there was a blackout and one of my osds remains down, I have noticed that the journal partition an data partion is not showed anymore so the device cannot mounted… 8 1145241856 sdh2 8 128 3906249728 sdi 8 129 3901005807 sdi1 8 1305241856 sdi2 8 144 3906249728 sdj 8 145 3901005807 sdj1 8 1465241856 sdj2 8 192 3906249728 sdm 8 176 3906249728 sdl 8 177 3901005807 sdl1 8 1785241856 sdl2 8 160 3906249728 sdk 8 161 3901005807 sdk1 8 1625241856 sdk2 2530 52428800 dm-0 25314194304 dm-1 2532 37588992 dm-2 the device is the /dev/sdm and the osd is the number 110, so what does that mean ? that I have lost everything in OSD 110? Thanks /dev/mapper/rhel-root 50G 4.4G 46G 9% / devtmpfs 126G 0 126G 0% /dev tmpfs 126G 92K 126G 1% /dev/shm tmpfs 126G 11M 126G 1% /run tmpfs 126G 0 126G 0% /sys/fs/cgroup /dev/sda1 494M 165M 330M 34% /boot /dev/sdj1 3.7T 220M 3.7T 1% /var/lib/ceph/osd/ceph-80 /dev/mapper/rhel-home 36G 49M 36G 1% /home /dev/sdg1 3.7T 256M 3.7T 1% /var/lib/ceph/osd/ceph-50 /dev/sdd1 3.7T 320M 3.7T 1% /var/lib/ceph/osd/ceph-20 /dev/sdc1 3.7T 257M 3.7T 1% /var/lib/ceph/osd/ceph-10 /dev/sdi1 3.7T 252M 3.7T 1% /var/lib/ceph/osd/ceph-70 /dev/sdl1 3.7T 216M 3.7T 1% /var/lib/ceph/osd/ceph-100 /dev/sdh1 3.7T 301M 3.7T 1% /var/lib/ceph/osd/ceph-60 /dev/sde1 3.7T 268M 3.7T 1% /var/lib/ceph/osd/ceph-30 /dev/sdf1 3.7T 299M 3.7T 1% /var/lib/ceph/osd/ceph-40 /dev/sdb1 3.7T 244M 3.7T 1% /var/lib/ceph/osd/ceph-0 /dev/sdk1 3.7T 240M 3.7T 1% /var/lib/ceph/osd/ceph-90 [root@capricornio ~]# 0 3.63osd.0 up 1 10 3.63osd.10 up 1 20 3.63osd.20 up 1 30 3.63osd.30 up 1 40 3.63osd.40 up 1 50 3.63osd.50 up 1 60 3.63osd.60 up 1 70 3.63osd.70 up 1 80 3.63osd.80 up 1 90 3.63osd.90 up 1 100 3.63osd.100 up 1 110 3.63osd.110 down0 [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] OSD Forece Removal
Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Thanks [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Forece Removal
Thanks stephane the thing is that those steps needs to be run in the node where the osd lives, I dont have that node any more since the operating Systems got corrupted so I Couldnt make it work :( Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 3:49 AM, Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr wrote: Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Fwd: OSD Forece Removal
Any idea how to forcé remove ? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 Begin forwarded message: From: Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr Date: March 20, 2015 at 3:49:11 AM CST To: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com Cc: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] OSD Forece Removal Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Forece Removal
Yes that exactly what I did but ceph osd tree still shows the osds Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 9:41 AM, John Spray john.sp...@redhat.commailto:john.sp...@redhat.com wrote: On 20/03/2015 15:23, Jesus Chavez (jeschave) wrote: Thanks stephane the thing is that those steps needs to be run in the node where the osd lives, I dont have that node any more since the operating Systems got corrupted so I Couldnt make it work :( Assuming the OSD is already down+out, you can skip straight to the Removing the OSD part[1] and run those commands from one of your mons. John 1. http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-the-osd ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Fwd: OSD Forece Removal
Maybe I should Edit the crushmap and delete osd... Is that a way yo force them? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 2:21 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Removing the OSD from the CRUSH map and deleting the auth key is how you force remove an OSD. The OSD can no longer participate in the cluster, even if it does come back to life. All clients forget about the OSD when the new CRUSH map is distributed. On Fri, Mar 20, 2015 at 11:19 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Any idea how to forcé remove ? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 Begin forwarded message: From: Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr Date: March 20, 2015 at 3:49:11 AM CST To: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com Cc: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] OSD Forece Removal Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks [cid:] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Forece Removal
This is the output if I try to remove from the crush map it says that a is already out… [root@capricornio ~]# ceph osd crush remove osd.29 device 'osd.29' does not appear in the crush map [root@capricornio ~]# [root@capricornio ~]# ceph osd tree | grep down # idweight type name up/down reweight 6 3.63osd.6 down0 16 3.63osd.16 down0 26 3.63osd.26 down0 36 3.63osd.36 down0 46 3.63osd.46 down0 56 3.63osd.56 down0 66 3.63osd.66 down0 76 3.63osd.76 down0 86 3.63osd.86 down0 96 3.63osd.96 down0 106 3.63osd.106 down0 116 3.63osd.116 down0 9 0 osd.9 down0 19 0 osd.19 down0 29 0 osd.29 down0 39 0 osd.39 down0 49 0 osd.49 down0 59 0 osd.59 down0 69 0 osd.69 down0 79 0 osd.79 down0 89 0 osd.89 down0 99 0 osd.99 down0 109 0 osd.109 down0 119 0 osd.119 down0 [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 20, 2015, at 4:13 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Does it show DNE in the entry? That stands for Does Not Exist. It will disappear on it's own after a while. I don't know what the timeout is, but they have always gone away within 24 hours. I've edited the CRUSH map before and I don't think it removed it when it was already DNE, I just had to wait for it to go away on it's own. On Fri, Mar 20, 2015 at 3:55 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Maybe I should Edit the crushmap and delete osd... Is that a way yo force them? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 2:21 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Removing the OSD from the CRUSH map and deleting the auth key is how you force remove an OSD. The OSD can no longer participate in the cluster, even if it does come back to life. All clients forget about the OSD when the new CRUSH map is distributed. On Fri, Mar 20, 2015 at 11:19 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Any idea how to forcé remove ? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 Begin forwarded message: From: Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr Date: March 20, 2015 at 3:49:11 AM CST To: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com Cc: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] OSD Forece Removal Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks [X] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [X] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp
Re: [ceph-users] OSD Forece Removal
thats what you sayd? [root@capricornio ~]# ceph auth del osd.9 entity osd.9 does not exist [root@capricornio ~]# ceph auth del osd.19 entity osd.19 does not exist [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 20, 2015, at 4:13 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Does it show DNE in the entry? That stands for Does Not Exist. It will disappear on it's own after a while. I don't know what the timeout is, but they have always gone away within 24 hours. I've edited the CRUSH map before and I don't think it removed it when it was already DNE, I just had to wait for it to go away on it's own. On Fri, Mar 20, 2015 at 3:55 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Maybe I should Edit the crushmap and delete osd... Is that a way yo force them? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 20, 2015, at 2:21 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Removing the OSD from the CRUSH map and deleting the auth key is how you force remove an OSD. The OSD can no longer participate in the cluster, even if it does come back to life. All clients forget about the OSD when the new CRUSH map is distributed. On Fri, Mar 20, 2015 at 11:19 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Any idea how to forcé remove ? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 Begin forwarded message: From: Stéphane DUGRAVOT stephane.dugra...@univ-lorraine.frmailto:stephane.dugra...@univ-lorraine.fr Date: March 20, 2015 at 3:49:11 AM CST To: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com Cc: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Subject: Re: [ceph-users] OSD Forece Removal Hi all, can anybody tell me how can I force delete osds? the thing is that one node got corrupted because of outage, so there is no way to get those osd up and back, is there anyway to force the removal from ceph-deploy node? Hi, Try manual : * http://ceph.com/docs/master/rados/operations/add-or-rm-osds/#removing-osds-manual Thanks [X] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [X] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Reclaim space from deleted files
Hi again everyone after some tests I have noticed that the data deleted on the filesystem doesnt really goes on the cluster... In the filesystem the uses space is 256MB and in ceph status shows us 25459MB thats 24GB so I was wondering if there is any way to reclaim space from deleted files or check why ceph cluster has 24GB used? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] ceph.conf
Thank you so much Thomas, Alexander! I really appreciate it Have a great time :) Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 18, 2015, at 5:37 AM, Alexandre DERUMIER aderum...@odiso.commailto:aderum...@odiso.com wrote: Hi, you don't need to defined ip and host for osd, but you need to defined monitor ips [osd] keyring = /var/lib/ceph/osd/ceph-$id/keyring [mon.0] host = node1 mon addr = X.X.X.X:6789 [mon.1] host = node2 mon addr = X.X.X.X:6789 [mon.2] host = node3 mon addr = X.X.X.X:6789 - Mail original - De: Jesus Chavez jesch...@cisco.commailto:jesch...@cisco.com À: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Envoyé: Mardi 17 Mars 2015 01:35:39 Objet: [ceph-users] ceph.conf Hi all I have seen that new versions of CEPH with new OS like RHEL7 and Cento7 doesn’t need information like mon.node1 and osd.0 etc.. anymore, can anybody tell me if is that for real? or do I need still need to write config like this: [osd.0] host = sagitario addr = 192.168.1.67 [mon.leo] host = leo mon addr = 192.168.1.81:6789 Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://Cisco.com Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click here for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Question Blackout
Hi everyone, I am ready to launch ceph on production but there is one thing that keeps on my mind... If there was a Blackout where all the ceph nodes went off what would really happen with the filesystem? It would get corrupt? Or ceph has any Kind of mechanism to survive to something like that? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] ceph.conf
Hi all I have seen that new versions of CEPH with new OS like RHEL7 and Cento7 doesn’t need information like mon.node1 and osd.0 etc.. anymore, can anybody tell me if is that for real? or do I need still need to write config like this: [osd.0] host = sagitario addr = 192.168.1.67 [mon.leo] host = leo mon addr = 192.168.1.81:6789 [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] HEALTH_WARN too few pgs per osd (0 min 20)
Hi all, does anybody know why still get WARN message status?
I don’t even have pools yet son I am not sure why is warning me…
[root@capricornio ceph-cluster]# ceph status
cluster d39f6247-1543-432d-9247-6c56f65bb6cd
health HEALTH_WARN too few pgs per osd (0 min 20)
monmap e1: 3 mons at
{capricornio=192.168.4.44:6789/0,geminis=192.168.4.37:6789/0,tauro=192.168.4.36:6789/0},
election epoch 46, quorum 0,1,2 tauro,geminis,capricornio
osdmap e956: 120 osds: 120 up, 120 in
pgmap v6252: 0 pgs, 0 pools, 0 bytes data, 0 objects
10177 MB used, 435 TB / 435 TB avail
[root@capricornio ceph-cluster]#
[root@capricornio ceph-cluster]# ceph osd lspools
[root@capricornio ceph-cluster]#
Thanks
[cid:image005.png@01D00809.A6D502D0]
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
[cid:image006.gif@01D00809.A6D502D0]
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] HEALTH_WARN too few pgs per osd (0 min 20)
yeah thats what I thought, I had the same WARN message with RHEL6.6 but I had a
pool and when I changed the value of pgs the message just gone...
[cid:image005.png@01D00809.A6D502D0]
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
[cid:image006.gif@01D00809.A6D502D0]
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
On Mar 15, 2015, at 9:23 AM, Jesus Chavez (jeschave)
jesch...@cisco.commailto:jesch...@cisco.com wrote:
Hi all, does anybody know why still get WARN message status?
I don’t even have pools yet son I am not sure why is warning me…
[root@capricornio ceph-cluster]# ceph status
cluster d39f6247-1543-432d-9247-6c56f65bb6cd
health HEALTH_WARN too few pgs per osd (0 min 20)
monmap e1: 3 mons at
{capricornio=192.168.4.44:6789/0,geminis=192.168.4.37:6789/0,tauro=192.168.4.36:6789/0},
election epoch 46, quorum 0,1,2 tauro,geminis,capricornio
osdmap e956: 120 osds: 120 up, 120 in
pgmap v6252: 0 pgs, 0 pools, 0 bytes data, 0 objects
10177 MB used, 435 TB / 435 TB avail
[root@capricornio ceph-cluster]#
[root@capricornio ceph-cluster]# ceph osd lspools
[root@capricornio ceph-cluster]#
Thanks
image005.png
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
image006.gif
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] osd goes down
Hi guys!
After get a cluster for 10 servers and build a image for a pool of 450TB, it
got sucked at the mkfs moment and what I noticed was that my entire cluster was
failing in a really weird way, some osds goes down and up from different nodes
and repeat so now I have a lot of PGs degraded and stuck :S I don’t know why
the cluster behave in that way (osds going up and down) do you think is a bug?
[root@capricornio ceph-cluster]# ceph status
cluster d39f6247-1543-432d-9247-6c56f65bb6cd
health HEALTH_WARN 109 pgs degraded; 251 pgs down; 1647 pgs peering; 2598
pgs stale; 1618 pgs stuck inactive; 1643 pgs stuck unclean; 109 pgs undersized;
1 requests are blocked 32 sec; recovery 13/1838 objects degraded (0.707%);
1/107 in osds are down
monmap e1: 3 mons at
{capricornio=192.168.4.44:6789/0,geminis=192.168.4.37:6789/0,tauro=192.168.4.36:6789/0},
election epoch 50, quorum 0,1,2 tauro,geminis,capricornio
osdmap e5275: 119 osds: 106 up, 107 in
pgmap v15826: 8192 pgs, 1 pools, 2016 MB data, 919 objects
48484 MB used, 388 TB / 388 TB avail
13/1838 objects degraded (0.707%)
2162 stale+active+clean
1031 peering
4274 active+clean
3 stale+remapped+peering
32 stale+active+undersized+degraded
43 stale+down+peering
4 remapped+peering
77 active+undersized+degraded
208 down+peering
358 stale+peering
[cid:image005.png@01D00809.A6D502D0]
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
[cid:image006.gif@01D00809.A6D502D0]
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Adding Monitor
So am I the firewall was the reason
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146tel:+52%2055%205267%203146
Mobile: +51 1 5538883255tel:+51%201%205538883255
CCIE - 44433
On Mar 13, 2015, at 3:30 PM, Andrija Panic
andrija.pa...@gmail.commailto:andrija.pa...@gmail.com wrote:
Check firewall - I hit this issue over and over again...
On 13 March 2015 at 22:25, Georgios Dimitrakakis
gior...@acmac.uoc.grmailto:gior...@acmac.uoc.gr wrote:
On an already available cluster I 've tried to add a new monitor!
I have used ceph-deploy mon create {NODE}
where {NODE}=the name of the node
and then I restarted the /etc/init.d/ceph service with a success at the node
where it showed that the monitor is running like:
# /etc/init.d/ceph restart
=== mon.jin ===
=== mon.jin ===
Stopping Ceph mon.jin on jin...kill 36388...done
=== mon.jin ===
Starting Ceph mon.jin on jin...
Starting ceph-create-keys on jin...
But checking the quorum it doesn't show the newly added monitor!
Plus ceph mon stat gives out only 1 monitor!!!
# ceph mon stat
e1: 1 mons at {fu=192.168.1.100:6789/0http://192.168.1.100:6789/0}, election
epoch 1, quorum 0 fu
Any ideas on what have I done wrong???
Regards,
George
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
--
Andrija Panić
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] [ERROR ] Some monitors have still not reached quorum
Hi all Does anybody know why am I having this error: [ceph_deploy.mon][WARNIN] waiting 20 seconds before retrying [ceph_deploy.mon][ERROR ] Some monitors have still not reached quorum: [ceph_deploy.mon][ERROR ] capricornio [ceph_deploy.mon][ERROR ] tauro [ceph_deploy.mon][ERROR ] aries the installation sequence was: ceph-deploy new aries capricornio tauro vi /my-cluster/ceph.conf [global] fsid = 66239891-8129-46e3-83cd-845f27af189b mon_initial_members = aries, capricornio, tauro mon_host = 192.168.4.35,192.168.4.44,192.168.4.36 auth_cluster_required = cephx auth_service_required = cephx auth_client_required = cephx filestore_xattr_use_omap = true osd pool default size = 2 public network = 192.168.4.0/25 ceph-deploy install aries capricornio tauro ceph-deploy mon create-initial thanks [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] OSD booting down
Hi all, after adding osds manually and reboot the server the osd didnt come up automatically am I missing something? Thanks Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Hi Robert yes I did disable completely actually with chkconfig off for not take the service up when booting, I have 2 networks 1 with internet for yum purposes and the network for the public network so before any configuration I specified on ceph.conf that public network but I am not sure if it is the cause for something... The thing is that I am not sure about which steps should run in the new monitor host and which one should I run in the initial monitor, it seems like step 3 and 4 that is generate keyring and mapping should be done in initial monitor server and also step 5 that is mkfs becuase if I try to run those steps in the new monitor host didnt work cause cant find the keys :( Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 7:54 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: If I remember right, the mon key has to be the same between all the mon hosts. I don't think I added an admin key to my second mon, it got all the other keys once it joined the mon closure. I do remember the join taking a while. Have you checked the firewall to make sure traffic is allowed? I don't remember if you said you checked it. Robert LeBlanc Sent from a mobile device please excuse any typos. On Mar 11, 2015 8:08 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thanks Steffen I have followed everything not sure what is going on, the mon keyring and client admin are individual? Per mon host? Or do I need to copy from the first initial mon node? Thanks again! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 11, 2015, at 6:28 PM, Steffen W Sørensen ste...@me.commailto:ste...@me.com wrote: On 12/03/2015, at 00.55, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: can anybody tell me a good blog link that explain how to add monitor? I have tried manually and also with ceph-deploy without success =( Dunno if these might help U: http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-a-monitor-manual http://cephnotes.ksperis.com/blog/2013/08/29/mon-failed-to-start /Steffen ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Great :) so just 1 point more, step 4 in adding monitors (Add the new monitor to the Monitor map.) this command actually runs in the new monitor right? Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 10:06 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Add the new monitor to the Monitor map. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Thanks I did everything as you mention and still have the same issue that hangs a lot: [root@tauro ~]# ceph status 2015-03-12 11:40:50.441084 7f6e20336700 0 -- :/1005688 192.168.4.35:6789/0 pipe(0x7f6e1c0239a0 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e1c023c30).fault 2015-03-12 11:40:53.441517 7f6e20235700 0 -- :/1005688 192.168.4.35:6789/0 pipe(0x7f6e1c00 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e1e90).fault 2015-03-12 11:40:57.443038 7f6e20336700 0 -- :/1005688 192.168.4.35:6789/0 pipe(0x7f6e100030e0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e10003370).fault ^CError connecting to cluster: InterruptedOrTimeoutError [root@tauro ~]# After I saw that my monmap doesn’t have the new entry: monmaptool: monmap file /tmp/monmap epoch 0 fsid ddb39f8d-c30f-4172-aeca-30e553a5e4f1 last_changed 2015-03-12 11:22:20.811525 created 2015-03-12 11:22:20.811525 0: 192.168.4.35:6789/0 mon.aries [root@tauro ~]# It only has the initial monitor (mon.aries) the new one is called (mon.tauro), the last command before “…ceph-tauro/done” was sudo ceph-mon --cluster ceph --mkfs -i tauro --keyring /tmp/ceph.mon.keyring --monmap /tmp/monmap” also I have tried to add to monmap with: [root@tauro ~]# ceph mon add tauro 192.168.4.36:6789 2015-03-12 11:45:26.807504 7fc53077a700 0 -- :/1007053 192.168.4.35:6789/0 pipe(0x7fc52c023970 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7fc52c023c00).fault 2015-03-12 11:45:29.807800 7fc530679700 0 -- :/1007053 192.168.4.35:6789/0 pipe(0x7fc52c00 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7fc52e90).fault ^CError connecting to cluster: InterruptedOrTimeoutError and also I tried from the initial monitor: [root@aries ~]# ceph mon add tauro 192.168.4.36:6789 2015-03-12 11:46:21.561523 7f7432ffd700 0 monclient: hunting for new mon and it hangs forever, I tried at the same time to get the service up on tauro (new mon): [root@tauro ~]# sudo /etc/init.d/ceph start mon.tauro === mon.tauro === Starting Ceph mon.tauro on tauro... Running as unit run-7463.service. Starting ceph-create-keys on tauro... [root@tauro ~]# and of course still have nothing =( those were my entire lines: vi /etc/ceph/ceph.conf i[global] fsid = ddb39f8d-c30f-4172-aeca-30e553a5e4f1 mon initial members = aries mon host = 192.168.4.35 ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *' ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow' ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring monmaptool --create --add aries 192.168.4.35 --fsid ddb39f8d-c30f-4172-aeca-30e553a5e4f1 /tmp/monmap sudo mkdir /var/lib/ceph/mon/ceph-aries ceph-mon --mkfs -i aries --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring sudo touch /var/lib/ceph/mon/ceph-aries/done sudo /etc/init.d/ceph start mon.aries scp /etc/ceph/ceph.conf root@tauro:/etc/ceph/ceph.conf scp /tmp/ceph.mon.keyring root@tauro:/tmp/ceph.mon.keyring scp /tmp/monmap root@tauro:/tmp/monmap mkdir /var/lib/ceph/mon/ceph-tauro sudo ceph-mon --cluster ceph --mkfs -i tauro --keyring /tmp/ceph.mon.keyring --monmap /tmp/monmap sudo ceph-mon -i tauro --mkfs --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring sudo touch /var/lib/ceph/mon/ceph-tauro/done sudo /etc/init.d/ceph start mon.tauro =( [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 12, 2015, at 10:33 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Great :) so just 1 point more, step 4 in adding monitors (Add the new monitor to the Monitor map.) this command actually runs in the new monitor right? Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 10:06 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Add the new monitor to the Monitor map. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Thank you Robert! Ill try ;) Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 11:03 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: That command (ceph mon add mon-id ip[:port]) can be run from any client in the cluster with the admin key, it is a general Ceph command. On Thu, Mar 12, 2015 at 10:33 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Great :) so just 1 point more, step 4 in adding monitors (Add the new monitor to the Monitor map.) this command actually runs in the new monitor right? Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 10:06 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Add the new monitor to the Monitor map. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Hi Steffen I already had them in my configuration I am stress now because it seems like none of the methods did help :( this is bad I think I am going to get back to rhel6.6 where xfs is a damn add on and I have to install from centos repo make ceph like patch :( but at last with RHEL6.6 work, Shame on RHEL7 the next time I will sell everything with ubuntu lol Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 1:56 PM, Steffen W Sørensen ste...@me.commailto:ste...@me.com wrote: On 12/03/2015, at 20.00, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thats what I thought and did actually the monmap and keyring were copied to the new monitor and there with 2 elements I did the mkfs thing and still have that Messages, do I need osd configured? Because I have non and I am not sure if it is requiered ... Also is weird that monmap is not taking the new monitor I think I should try to configure the 3 monitors as initial monitors an see how it goes Dunno about your config, but I seem to remember when I decommissioned one mon instance and addition of a new on another node that I needed to have mon.id section in ceph.conf inorder to be able to start the monitor. ceph.conf snippet: [osd] osd mount options xfs = rw,noatime,nobarrier,logbsize=256k,logbufs=8,allocsize=4M,attr2,delaylog,inode64,noquota keyring = /var/lib/ceph/osd/ceph-$id/keyring ; Tuning ;# By default, Ceph makes 3 replicas of objects. If you want to make four ;# copies of an object the default value--a primary copy and three replica ;# copies--reset the default values as shown in 'osd pool default size'. ;# If you want to allow Ceph to write a lesser number of copies in a degraded ;# state, set 'osd pool default min size' to a number less than the ;# 'osd pool default size' value. osd pool default size = 2 # Write an object 2 times. osd pool default min size = 1 # Allow writing one copy in a degraded state. ;# Ensure you have a realistic number of placement groups. We recommend ;# approximately 100 per OSD. E.g., total number of OSDs multiplied by 100 ;# divided by the number of replicas (i.e., osd pool default size). So for ;# 10 OSDs and osd pool default size = 3, we'd recommend approximately ;# (100 * 10) / 3 = 333. ;# got 24 OSDs = 1200 pg, but this is not a full production site, so let's settle for 1024 to lower cpu load osd pool default pg num = 1024 osd pool default pgp num = 1024 client cache size = 131072 osd client op priority = 40 osd op threads = 8 osd client message size cap = 512 filestore min sync interval = 10 filestore max sync interval = 60 ;filestore queue max bytes = 10485760 ;filestore queue max ops = 50 ;filestore queue committing max ops = 500 ;filestore queue committing max bytes = 104857600 ;filestore op threads = 2 recovery max active = 2 recovery op priority = 30 osd max backfills = 2 ; Journal Tuning journal size = 5120 ;journal max write bytes = 1073714824 ;journal max write entries = 1 ;journal queue max ops = 5 ;journal queue max bytes = 1048576 [mon.0] host = node4 mon addr = 10.0.3.4:6789 [mon.1] host = node2 mon addr = 10.0.3.2:6789 [mon.2] host = node1 mon addr = 10.0.3.1:6789 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
This is awkard Robert all this time was the firewall :( I cant believe I spent 2 days trying to figure out :(. Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 2:48 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: I'm not sure why you are having such a hard time. I added monitors (and removed them) on CentOS 7 by following what I had. The thing that kept tripping me up was firewalld. Once I either shut it off or created a service for Ceph, it worked fine. What is in in /var/log/ceph/ceph-mon.tauro.log when it is hunting for a monitor? On Thu, Mar 12, 2015 at 2:31 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Hi Steffen I already had them in my configuration emoji_u1f61e.png I am stress now because it seems like none of the methods did help :( this is bad I think I am going to get back to rhel6.6 where xfs is a damn add on and I have to install from centos repo make ceph like patch :( but at last with RHEL6.6 work, Shame on RHEL7 the next time I will sell everything with ubuntu lol Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 1:56 PM, Steffen W Sørensen ste...@me.commailto:ste...@me.com wrote: On 12/03/2015, at 20.00, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thats what I thought and did actually the monmap and keyring were copied to the new monitor and there with 2 elements I did the mkfs thing and still have that Messages, do I need osd configured? Because I have non and I am not sure if it is requiered ... Also is weird that monmap is not taking the new monitor I think I should try to configure the 3 monitors as initial monitors an see how it goes Dunno about your config, but I seem to remember when I decommissioned one mon instance and addition of a new on another node that I needed to have mon.id section in ceph.conf inorder to be able to start the monitor. ceph.conf snippet: [osd] osd mount options xfs = rw,noatime,nobarrier,logbsize=256k,logbufs=8,allocsize=4M,attr2,delaylog,inode64,noquota keyring = /var/lib/ceph/osd/ceph-$id/keyring ; Tuning ;# By default, Ceph makes 3 replicas of objects. If you want to make four ;# copies of an object the default value--a primary copy and three replica ;# copies--reset the default values as shown in 'osd pool default size'. ;# If you want to allow Ceph to write a lesser number of copies in a degraded ;# state, set 'osd pool default min size' to a number less than the ;# 'osd pool default size' value. osd pool default size = 2 # Write an object 2 times. osd pool default min size = 1 # Allow writing one copy in a degraded state. ;# Ensure you have a realistic number of placement groups. We recommend ;# approximately 100 per OSD. E.g., total number of OSDs multiplied by 100 ;# divided by the number of replicas (i.e., osd pool default size). So for ;# 10 OSDs and osd pool default size = 3, we'd recommend approximately ;# (100 * 10) / 3 = 333. ;# got 24 OSDs = 1200 pg, but this is not a full production site, so let's settle for 1024 to lower cpu load osd pool default pg num = 1024 osd pool default pgp num = 1024 client cache size = 131072 osd client op priority = 40 osd op threads = 8 osd client message size cap = 512 filestore min sync interval = 10 filestore max sync interval = 60 ;filestore queue max bytes = 10485760 ;filestore queue max ops = 50 ;filestore queue committing max ops = 500 ;filestore queue committing max bytes = 104857600 ;filestore op threads = 2 recovery max active = 2 recovery op priority = 30 osd max backfills = 2 ; Journal Tuning journal size = 5120 ;journal max write bytes = 1073714824 ;journal max write entries = 1 ;journal queue max ops = 5 ;journal queue max bytes = 1048576 [mon.0] host = node4 mon addr = 10.0.3.4:6789http://10.0.3.4:6789 [mon.1] host = node2 mon addr = 10.0.3.2:6789http://10.0.3.2:6789 [mon.2] host = node1 mon addr = 10.0.3.1:6789http://10.0.3.1:6789 ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Could not find keyring file: /etc/ceph/ceph.client.admin.keyring
Ok guys I decided to get back to ceph-deploy after mon create command I have got this: [root@capricornio ~]# ceph-deploy gatherkeys capricornio [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.22): /usr/bin/ceph-deploy gatherkeys capricornio [ceph_deploy.gatherkeys][DEBUG ] Checking capricornio for /etc/ceph/ceph.client.admin.keyring [capricornio][DEBUG ] connected to host: capricornio [capricornio][DEBUG ] detect platform information from remote host [capricornio][DEBUG ] detect machine type [capricornio][DEBUG ] fetch remote file [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on capricornio [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: /etc/ceph/ceph.client.admin.keyring on host capricornio [root@capricornio ~]# Anybody can help me getting new keyring form monmap existing in the system? Thanks [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 12, 2015, at 10:33 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Great :) so just 1 point more, step 4 in adding monitors (Add the new monitor to the Monitor map.) this command actually runs in the new monitor right? Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 10:06 AM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: Add the new monitor to the Monitor map. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Thats what I thought and did actually the monmap and keyring were copied to the new monitor and there with 2 elements I did the mkfs thing and still have that Messages, do I need osd configured? Because I have non and I am not sure if it is requiered ... Also is weird that monmap is not taking the new monitor I think I should try to configure the 3 monitors as initial monitors an see how it goes Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 12:44 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: I might be missing something, but is sounds like you already have a monitor up and running. If you create a new key, the new monitor won't be able to auth to the existing one. You need to get the monitor key from your existing monitor and use that for the second (and third) monitor. Look at step #3 at http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-a-monitor-manual. You only have to build a fresh keyring for your first monitor. Your subsequent monitors will use the first monitor's keyring. On Thu, Mar 12, 2015 at 11:48 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thanks I did everything as you mention and still have the same issue that hangs a lot: [root@tauro ~]# ceph status 2015-03-12 11:40:50.441084 7f6e20336700 0 -- :/1005688 192.168.4.35:6789/0http://192.168.4.35:6789/0 pipe(0x7f6e1c0239a0 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e1c023c30).fault 2015-03-12 11:40:53.441517 7f6e20235700 0 -- :/1005688 192.168.4.35:6789/0http://192.168.4.35:6789/0 pipe(0x7f6e1c00 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e1e90).fault 2015-03-12 11:40:57.443038 7f6e20336700 0 -- :/1005688 192.168.4.35:6789/0http://192.168.4.35:6789/0 pipe(0x7f6e100030e0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f6e10003370).fault ^CError connecting to cluster: InterruptedOrTimeoutError [root@tauro ~]# After I saw that my monmap doesn’t have the new entry: monmaptool: monmap file /tmp/monmap epoch 0 fsid ddb39f8d-c30f-4172-aeca-30e553a5e4f1 last_changed 2015-03-12 11:22:20.811525 created 2015-03-12 11:22:20.811525 0: 192.168.4.35:6789/0http://192.168.4.35:6789/0 mon.aries [root@tauro ~]# It only has the initial monitor (mon.aries) the new one is called (mon.tauro), the last command before “…ceph-tauro/done” was sudo ceph-mon --cluster ceph --mkfs -i tauro --keyring /tmp/ceph.mon.keyring --monmap /tmp/monmap” also I have tried to add to monmap with: [root@tauro ~]# ceph mon add tauro 192.168.4.36:6789http://192.168.4.36:6789 2015-03-12 11:45:26.807504 7fc53077a700 0 -- :/1007053 192.168.4.35:6789/0http://192.168.4.35:6789/0 pipe(0x7fc52c023970 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7fc52c023c00).fault 2015-03-12 11:45:29.807800 7fc530679700 0 -- :/1007053 192.168.4.35:6789/0http://192.168.4.35:6789/0 pipe(0x7fc52c00 sd=3 :0 s=1 pgs=0 cs=0 l=1 c=0x7fc52e90).fault ^CError connecting to cluster: InterruptedOrTimeoutError and also I tried from the initial monitor: [root@aries ~]# ceph mon add tauro 192.168.4.36:6789http://192.168.4.36:6789 2015-03-12 11:46:21.561523 7f7432ffd700 0 monclient: hunting for new mon and it hangs forever, I tried at the same time to get the service up on tauro (new mon): [root@tauro ~]# sudo /etc/init.d/ceph start mon.tauro === mon.tauro === Starting Ceph mon.tauro on tauro... Running as unit run-7463.service. Starting ceph-create-keys on tauro... [root@tauro ~]# and of course still have nothing =( those were my entire lines: vi /etc/ceph/ceph.conf i[global] fsid = ddb39f8d-c30f-4172-aeca-30e553a5e4f1 mon initial members = aries mon host = 192.168.4.35 ceph-authtool --create-keyring /tmp/ceph.mon.keyring --gen-key -n mon. --cap mon 'allow *' ceph-authtool --create-keyring /etc/ceph/ceph.client.admin.keyring --gen-key -n client.admin --set-uid=0 --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow' ceph-authtool /tmp/ceph.mon.keyring --import-keyring /etc/ceph/ceph.client.admin.keyring monmaptool --create --add aries 192.168.4.35 --fsid ddb39f8d-c30f-4172-aeca-30e553a5e4f1 /tmp/monmap sudo mkdir /var/lib/ceph/mon/ceph-aries ceph-mon --mkfs -i aries --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring sudo touch /var/lib/ceph/mon/ceph-aries/done sudo /etc/init.d/ceph start mon.aries scp /etc/ceph/ceph.conf root@tauro:/etc/ceph/ceph.conf scp /tmp/ceph.mon.keyring root@tauro:/tmp/ceph.mon.keyring scp /tmp/monmap root@tauro:/tmp/monmap mkdir /var/lib/ceph/mon/ceph-tauro sudo ceph-mon --cluster ceph --mkfs -i tauro --keyring /tmp/ceph.mon.keyring --monmap /tmp/monmap sudo ceph-mon -i tauro --mkfs --monmap /tmp/monmap --keyring /tmp/ceph.mon.keyring sudo touch /var/lib/ceph/mon/ceph-tauro/done sudo /etc/init.d/ceph start mon.tauro =( image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch
Re: [ceph-users] Add monitor unsuccesful
Jajaja :) damn fire wall :P well thank you for your patience! Have a great day Robert and Steffen Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 3:20 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: We all get burned by the firewall at one time or another. Hence the name 'fire'wall! :) I'm glad you got it working. On Thu, Mar 12, 2015 at 2:53 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: This is awkard Robert all this time was the firewall :( I cant believe I spent 2 days trying to figure out :(. Thank you so much! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 2:48 PM, Robert LeBlanc rob...@leblancnet.usmailto:rob...@leblancnet.us wrote: I'm not sure why you are having such a hard time. I added monitors (and removed them) on CentOS 7 by following what I had. The thing that kept tripping me up was firewalld. Once I either shut it off or created a service for Ceph, it worked fine. What is in in /var/log/ceph/ceph-mon.tauro.log when it is hunting for a monitor? On Thu, Mar 12, 2015 at 2:31 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Hi Steffen I already had them in my configuration emoji_u1f61e.png I am stress now because it seems like none of the methods did help :( this is bad I think I am going to get back to rhel6.6 where xfs is a damn add on and I have to install from centos repo make ceph like patch :( but at last with RHEL6.6 work, Shame on RHEL7 the next time I will sell everything with ubuntu lol Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 12, 2015, at 1:56 PM, Steffen W Sørensen ste...@me.commailto:ste...@me.com wrote: On 12/03/2015, at 20.00, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thats what I thought and did actually the monmap and keyring were copied to the new monitor and there with 2 elements I did the mkfs thing and still have that Messages, do I need osd configured? Because I have non and I am not sure if it is requiered ... Also is weird that monmap is not taking the new monitor I think I should try to configure the 3 monitors as initial monitors an see how it goes Dunno about your config, but I seem to remember when I decommissioned one mon instance and addition of a new on another node that I needed to have mon.id section in ceph.conf inorder to be able to start the monitor. ceph.conf snippet: [osd] osd mount options xfs = rw,noatime,nobarrier,logbsize=256k,logbufs=8,allocsize=4M,attr2,delaylog,inode64,noquota keyring = /var/lib/ceph/osd/ceph-$id/keyring ; Tuning ;# By default, Ceph makes 3 replicas of objects. If you want to make four ;# copies of an object the default value--a primary copy and three replica ;# copies--reset the default values as shown in 'osd pool default size'. ;# If you want to allow Ceph to write a lesser number of copies in a degraded ;# state, set 'osd pool default min size' to a number less than the ;# 'osd pool default size' value. osd pool default size = 2 # Write an object 2 times. osd pool default min size = 1 # Allow writing one copy in a degraded state. ;# Ensure you have a realistic number of placement groups. We recommend ;# approximately 100 per OSD. E.g., total number of OSDs multiplied by 100 ;# divided by the number of replicas (i.e., osd pool default size). So for ;# 10 OSDs and osd pool default size = 3, we'd recommend approximately ;# (100 * 10) / 3 = 333. ;# got 24 OSDs = 1200 pg, but this is not a full production site, so let's settle for 1024 to lower cpu load osd pool default pg num = 1024 osd pool default pgp num = 1024 client cache size = 131072 osd client op priority = 40 osd op threads = 8 osd client message size cap = 512 filestore min sync interval = 10 filestore max sync interval = 60 ;filestore queue max bytes = 10485760 ;filestore queue max ops = 50 ;filestore queue committing max ops = 500 ;filestore queue committing max bytes = 104857600 ;filestore op threads = 2 recovery max active = 2 recovery op priority = 30 osd max backfills = 2 ; Journal Tuning journal size = 5120 ;journal max write bytes = 1073714824 ;journal max write entries = 1 ;journal queue max ops = 5 ;journal queue max bytes = 1048576 [mon.0] host = node4 mon addr = 10.0.3.4:6789http://10.0.3.4:6789 [mon.1] host = node2 mon addr = 10.0.3.2:6789http://10.0.3.2:6789 [mon.2] host = node1 mon addr = 10.0.3.1:6789http://10.0.3.1:6789
[ceph-users] Add monitor unsuccesful
can anybody tell me a good blog link that explain how to add monitor? I have tried manually and also with ceph-deploy without success =( Help [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] hang osd --zap-disk
I don’t know what is going on =( the system hangs with the message below after commaand ceph-deploy osd --zap-disk create tauro:sdb” [tauro][WARNING] No data was received after 300 seconds, disconnecting... [ceph_deploy.osd][DEBUG ] Host tauro is now ready for osd use. [ceph_deploy.conf][DEBUG ] found configuration file at: /root/.cephdeploy.conf [ceph_deploy.cli][INFO ] Invoked (1.5.22): /usr/bin/ceph-deploy osd activate tauro:sdb1 [ceph_deploy.osd][DEBUG ] Activating cluster ceph disks tauro:/dev/sdb1: [tauro][DEBUG ] connection detected need for sudo [tauro][DEBUG ] connected to host: tauro [tauro][DEBUG ] detect platform information from remote host [tauro][DEBUG ] detect machine type [ceph_deploy.osd][INFO ] Distro info: Red Hat Enterprise Linux Server 7.1 Maipo [ceph_deploy.osd][DEBUG ] activating host tauro disk /dev/sdb1 [ceph_deploy.osd][DEBUG ] will use init type: sysvinit [tauro][INFO ] Running command: sudo ceph-disk -v activate --mark-init sysvinit --mount /dev/sdb1 [tauro][WARNING] INFO:ceph-disk:Running command: /sbin/blkid -p -s TYPE -ovalue -- /dev/sdb1 [tauro][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_mount_options_xfs [tauro][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-conf --cluster=ceph --name=osd. --lookup osd_fs_mount_options_xfs [tauro][WARNING] DEBUG:ceph-disk:Mounting /dev/sdb1 on /var/lib/ceph/tmp/mnt.lNpFro with options noatime,inode64 [tauro][WARNING] INFO:ceph-disk:Running command: /usr/bin/mount -t xfs -o noatime,inode64 -- /dev/sdb1 /var/lib/ceph/tmp/mnt.lNpFro [tauro][WARNING] DEBUG:ceph-disk:Cluster uuid is fc72a252-15be-40e9-9de1-34593be5668a [tauro][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph-osd --cluster=ceph --show-config-value=fsid [tauro][WARNING] DEBUG:ceph-disk:Cluster name is ceph [tauro][WARNING] DEBUG:ceph-disk:OSD uuid is bf192166-86e9-4c68-9bff-7ced1c9ba8ee [tauro][WARNING] DEBUG:ceph-disk:Allocating OSD id... [tauro][WARNING] INFO:ceph-disk:Running command: /usr/bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring osd create --concise bf192166-86e9-4c68-9bff-7ced1c9ba8ee [tauro][WARNING] 2015-03-11 17:49:31.782184 7f9cf05a8700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9cec0253f0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cec025680).fault [tauro][WARNING] 2015-03-11 17:49:35.782524 7f9cf04a7700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9cec00 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cee90).fault [tauro][WARNING] 2015-03-11 17:49:37.781846 7f9cf05a8700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9ce00030e0 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9ce0003370).fault [tauro][WARNING] 2015-03-11 17:49:41.782566 7f9cf04a7700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9cec00 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cee90).fault [tauro][WARNING] 2015-03-11 17:49:43.782303 7f9cf05a8700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9ce00031b0 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9ce00025d0).fault [tauro][WARNING] 2015-03-11 17:49:47.784627 7f9cf04a7700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9cec00 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9cee90).fault [tauro][WARNING] 2015-03-11 17:49:49.782712 7f9cf05a8700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9ce00031b0 sd=5 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9ce0002c60).fault [tauro][WARNING] 2015-03-11 17:49:53.784690 7f9cf04a7700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9ce0003fb0 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9ce0004240).fault [tauro][WARNING] 2015-03-11 17:49:55.783248 7f9cf05a8700 0 -- :/1015927 192.168.4.35:6789/0 pipe(0x7f9ce0004930 sd=4 :0 s=1 pgs=0 cs=0 l=1 c=0x7f9ce0004bc0) [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Add monitor unsuccesful
can anybody tell me a good blog link that explain how to add monitor? I have tried manually and also with ceph-deploy without success =( Help [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Adding Monitor Stuck
I am really stuck adding second monitor =(, ceph-deploy mon create seems to
finish with some error like monitor may not be able to form quorum and they are
not definite in mon initial…
I have found there is a way to get it work and is doing the next commands:
ceph mon add tauro 192.168.4.35:6789
but this is weird because seems to be a command that you usually run after mkfs
something like this (do ceph-mon -i {mon-id} --mkfs --monmap
{tmp}/{map-filename} --keyring {tmp}/{key-filename}) :@ but that depends on
monmap and keyring things that you are not able to do in the “new monitor”
since it has nothing =( so even the manual way that if you follow the steps you
get lost because you don’t really know which command is for which server.
Also it says that you should star the new monitor while the “add command” is
hunting a mon client, but that depends on monmap and keyring again things that
you don’t have in the new server…
=( Im getting crazy can anybody explain how does really work?
Thanks
[cid:image005.png@01D00809.A6D502D0]
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
[cid:image006.gif@01D00809.A6D502D0]
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Add monitor unsuccesful
Thanks Steffen I have followed everything not sure what is going on, the mon keyring and client admin are individual? Per mon host? Or do I need to copy from the first initial mon node? Thanks again! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 11, 2015, at 6:28 PM, Steffen W Sørensen ste...@me.commailto:ste...@me.com wrote: On 12/03/2015, at 00.55, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: can anybody tell me a good blog link that explain how to add monitor? I have tried manually and also with ceph-deploy without success =( Dunno if these might help U: http://ceph.com/docs/master/rados/operations/add-or-rm-mons/#adding-a-monitor-manual http://cephnotes.ksperis.com/blog/2013/08/29/mon-failed-to-start /Steffen ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] New eu.ceph.com mirror machine
So EPEL is not requiered? Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 9, 2015, at 8:58 AM, HEWLETT, Paul (Paul)** CTR ** paul.hewl...@alcatel-lucent.commailto:paul.hewl...@alcatel-lucent.com wrote: Hi Wildo It seems that your move coincided with yet another change in the EPEL repo. For anyone who is interested, I fixed this by: 1. ensuring that check_obsoletes=1 is in /etc/yum/pluginconf.d/priorities.conf 2. Install libunwind explicitly: yum install libunwind 3. Install ceph with epel disabled: yum install --disablerepo=epel ceph Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 13:43 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com mirror machine On 03/09/2015 02:27 PM, HEWLETT, Paul (Paul)** CTR ** wrote: When did you make the change? Yesterday It worked on Friday albeit with these extra lines in ceph.repo: [Ceph-el7] name=Ceph-el7 baseurl=http://eu.ceph.com/rpms/rhel7/noarch/ enabled=1 gpgcheck=0 which I removed when I discovered this no longer existed. Ah, I think I know. The rsync script probably didn't clean up those old directories, since they don't exist here either: http://ceph.com/rpms/rhel7/noarch/ That caused some confusion since this machine is a fresh sync from ceph.comhttp://ceph.com Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 12:15 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com mirror machine On 03/09/2015 12:54 PM, HEWLETT, Paul (Paul)** CTR ** wrote: Hi Wildo Has something broken with this move? The following has worked for me repeatedly over the last 2 months: It shouldn't have broken anything, but you never know. The machine rsyncs the data from ceph.comhttp://ceph.com directly. The directories you are pointing at do exist and contain data. Anybody else noticing something? This a.m. I tried to install ceph using the following repo file: [root@citrus ~]# cat /etc/yum.repos.d/ceph.repo [ceph] name=Ceph packages for $basearch baseurl=http://ceph.com/rpm-giant/rhel7/$basearch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-noarch] name=Ceph noarch packages baseurl=http://ceph.com/rpm-giant/rhel7/noarch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-source] name=Ceph source packages baseurl=http://ceph.com/rpm-giant/rhel7/SRPMS enabled=0 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc and ceph now fails to install: msg: Error: Package: 1:ceph-0.87.1-0.el7.x86_64 (ceph) Requires: python-ceph = 1:0.87.1-0.el7 Available: 1:python-ceph-0.86-0.el7.x86_64 (ceph) python-ceph = 1:0.86-0.el7 Available: 1:python-ceph-0.87-0.el7.x86_64 (ceph) python-ceph = 1:0.87-0.el7 Available: 1:python-ceph-0.87.1-0.el7.x86_64 (ceph) python-ceph = 1:0.87.1-0.el7 Error: Package: 1:ceph-common-0.87.1-0.el7.x86_64 (ceph) Requires: python-ceph = 1:0.87.1-0.el7 Available: 1:python-ceph-0.86-0.el7.x86_64 (ceph) python-ceph = 1:0.86-0.el7 Available: 1:python-ceph-0.87-0.el7.x86_64 (ceph) python-ceph = 1:0.87-0.el7 Available: 1:python-ceph-0.87.1-0.el7.x86_64 (ceph) python-ceph = 1:0.87.1-0.el7 Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: ceph-users [ceph-users-boun...@lists.ceph.commailto:ceph-users-boun...@lists.ceph.com] on behalf of Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 11:15 To: ceph-users Subject: [ceph-users] New eu.ceph.comhttp://eu.ceph.com mirror machine Hi, Since the recent reports of rsync failing on eu.ceph.comhttp://eu.ceph.com I moved eu.ceph.comhttp://eu.ceph.com to a new machine. It went from physical to a KVM VM backed by RBD, so it's now running on Ceph. URLs or rsync paths haven't changed, it's still eu.ceph.comhttp://eu.ceph.com and available over IPv4 and IPv6. This Virtual Machine is dedicated for running eu.ceph.comhttp://eu.ceph.com, so hopefully rsync won't fail anymore. -- Wido den Hollander 42on B.V. Ceph trainer and consultant Phone: +31 (0)20
Re: [ceph-users] New eu.ceph.com mirror machine
thanks! but I still don’t get it very well what can I do to install libunwind? [root@aries ~]# yum install libunwind Loaded plugins: langpacks, priorities, product-id, subscription-manager 7 packages excluded due to repository priority protections No package libunwind available. Error: Nothing to do [root@aries ~]# [main] enabled = 1 check_obsoletes=1 ~ ~ ~ [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 10, 2015, at 7:27 AM, HEWLETT, Paul (Paul)** CTR ** paul.hewl...@alcatel-lucent.commailto:paul.hewl...@alcatel-lucent.com wrote: Hi Jesus EPEL is required for the libunwind library. If libunwind is copied to the ceph repo then EPEL would not be required. Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Jesus Chavez (jeschave) [jesch...@cisco.commailto:jesch...@cisco.com] Sent: 10 March 2015 12:15 To: HEWLETT, Paul (Paul)** CTR ** Cc: Wido den Hollander; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com mirror machine So EPEL is not requiered? Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 9, 2015, at 8:58 AM, HEWLETT, Paul (Paul)** CTR ** paul.hewl...@alcatel-lucent.commailto:paul.hewl...@alcatel-lucent.com wrote: Hi Wildo It seems that your move coincided with yet another change in the EPEL repo. For anyone who is interested, I fixed this by: 1. ensuring that check_obsoletes=1 is in /etc/yum/pluginconf.d/priorities.conf 2. Install libunwind explicitly: yum install libunwind 3. Install ceph with epel disabled: yum install --disablerepo=epel ceph Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 13:43 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com/ mirror machine On 03/09/2015 02:27 PM, HEWLETT, Paul (Paul)** CTR ** wrote: When did you make the change? Yesterday It worked on Friday albeit with these extra lines in ceph.repo: [Ceph-el7] name=Ceph-el7 baseurl=http://eu.ceph.com/rpms/rhel7/noarch/ enabled=1 gpgcheck=0 which I removed when I discovered this no longer existed. Ah, I think I know. The rsync script probably didn't clean up those old directories, since they don't exist here either: http://ceph.com/rpms/rhel7/noarch/ That caused some confusion since this machine is a fresh sync from ceph.comhttp://ceph.com/ Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 12:15 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com/ mirror machine On 03/09/2015 12:54 PM, HEWLETT, Paul (Paul)** CTR ** wrote: Hi Wildo Has something broken with this move? The following has worked for me repeatedly over the last 2 months: It shouldn't have broken anything, but you never know. The machine rsyncs the data from ceph.comhttp://ceph.com/ directly. The directories you are pointing at do exist and contain data. Anybody else noticing something? This a.m. I tried to install ceph using the following repo file: [root@citrus ~]# cat /etc/yum.repos.d/ceph.repo [ceph] name=Ceph packages for $basearch baseurl=http://ceph.com/rpm-giant/rhel7/$basearch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-noarch] name=Ceph noarch packages baseurl=http://ceph.com/rpm-giant/rhel7/noarch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-source] name=Ceph source packages baseurl=http://ceph.com/rpm-giant/rhel7/SRPMS enabled=0 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc and ceph now
Re: [ceph-users] New eu.ceph.com mirror machine
Or maybe installing the RPM directly from: http://www.mirrorservice.org/sites/ceph.com/rpm-firefly/rhel7/x86_64/ libunwind-1.1-3.el7.x86_64.rpmhttp://www.mirrorservice.org/sites/ceph.com/rpm-firefly/rhel7/x86_64/libunwind-1.1-3.el7.x86_64.rpm but this is not for Giant seems to be for firefly =( Rgards [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 10, 2015, at 7:27 AM, HEWLETT, Paul (Paul)** CTR ** paul.hewl...@alcatel-lucent.commailto:paul.hewl...@alcatel-lucent.com wrote: Hi Jesus EPEL is required for the libunwind library. If libunwind is copied to the ceph repo then EPEL would not be required. Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Jesus Chavez (jeschave) [jesch...@cisco.commailto:jesch...@cisco.com] Sent: 10 March 2015 12:15 To: HEWLETT, Paul (Paul)** CTR ** Cc: Wido den Hollander; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com mirror machine So EPEL is not requiered? Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 9, 2015, at 8:58 AM, HEWLETT, Paul (Paul)** CTR ** paul.hewl...@alcatel-lucent.commailto:paul.hewl...@alcatel-lucent.com wrote: Hi Wildo It seems that your move coincided with yet another change in the EPEL repo. For anyone who is interested, I fixed this by: 1. ensuring that check_obsoletes=1 is in /etc/yum/pluginconf.d/priorities.conf 2. Install libunwind explicitly: yum install libunwind 3. Install ceph with epel disabled: yum install --disablerepo=epel ceph Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 13:43 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com/ mirror machine On 03/09/2015 02:27 PM, HEWLETT, Paul (Paul)** CTR ** wrote: When did you make the change? Yesterday It worked on Friday albeit with these extra lines in ceph.repo: [Ceph-el7] name=Ceph-el7 baseurl=http://eu.ceph.com/rpms/rhel7/noarch/ enabled=1 gpgcheck=0 which I removed when I discovered this no longer existed. Ah, I think I know. The rsync script probably didn't clean up those old directories, since they don't exist here either: http://ceph.com/rpms/rhel7/noarch/ That caused some confusion since this machine is a fresh sync from ceph.comhttp://ceph.com/ Regards Paul Hewlett Senior Systems Engineer Velocix, Cambridge Alcatel-Lucent t: +44 1223 435893 m: +44 7985327353 From: Wido den Hollander [w...@42on.commailto:w...@42on.com] Sent: 09 March 2015 12:15 To: HEWLETT, Paul (Paul)** CTR **; ceph-users Subject: Re: [ceph-users] New eu.ceph.comhttp://eu.ceph.com/ mirror machine On 03/09/2015 12:54 PM, HEWLETT, Paul (Paul)** CTR ** wrote: Hi Wildo Has something broken with this move? The following has worked for me repeatedly over the last 2 months: It shouldn't have broken anything, but you never know. The machine rsyncs the data from ceph.comhttp://ceph.com/ directly. The directories you are pointing at do exist and contain data. Anybody else noticing something? This a.m. I tried to install ceph using the following repo file: [root@citrus ~]# cat /etc/yum.repos.d/ceph.repo [ceph] name=Ceph packages for $basearch baseurl=http://ceph.com/rpm-giant/rhel7/$basearch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-noarch] name=Ceph noarch packages baseurl=http://ceph.com/rpm-giant/rhel7/noarch enabled=1 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc [ceph-source] name=Ceph source packages baseurl=http://ceph.com/rpm-giant/rhel7/SRPMS enabled=0 priority=2 gpgcheck=1 type=rpm-md gpgkey=https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc and ceph now fails to install: msg: Error: Package: 1:ceph-0.87.1-0.el7
[ceph-users] Now it seems that could not find keyring
what is going on with ceph? [ceph_deploy.gatherkeys][WARNIN] Unable to find /etc/ceph/ceph.client.admin.keyring on aries [ceph_deploy][ERROR ] KeyNotFoundError: Could not find keyring file: /etc/ceph/ceph.client.admin.keyring on host aries =( gosh Jesus Chavez [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Ceph repo - RSYNC?
Hi David also for the Calamari or gui monitoring interface is there any way to get user account and passwd of inktank since the repo to install Calamari seems to be only for people inside of inktank Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 8, 2015, at 10:38 AM, David Moreau Simard dmsim...@iweb.commailto:dmsim...@iweb.com wrote: Hi, With the help of Inktank we have been providing a Ceph mirror at ceph.mirror.iweb.cahttp://ceph.mirror.iweb.ca. Quick facts: - Located on the east coast of Canada (Montreal, Quebec) - Syncs every four hours directly off of the official repositories - Available over http (http://ceph.mirror.iweb.ca/) and rsync (rsync://mirror.iweb.ca/ceph) We're working on a brand new, faster and improved infrastructure for all of our mirrors and it will be backed by Ceph.. So the Ceph mirror will soon be stored on a Ceph cluster :) Feel free to use it ! -- David Moreau Simard On 2015-03-05, 1:14 PM, Brian Rak b...@gameservers.commailto:b...@gameservers.com wrote: Do any of the Ceph repositories run rsync? We generally mirror the repository locally so we don't encounter any unexpected upgrades. eu.ceph.comhttp://eu.ceph.com used to run this, but it seems to be down now. # rsync rsync://eu.ceph.com rsync: failed to connect to eu.ceph.comhttp://eu.ceph.com: Connection refused (111) rsync error: error in socket IO (code 10) at clientserver.c(124) [receiver=3.0.6] ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Remains down
Yes actually I did it before ceph deployment. Also the filesystem is mounted =( [root@geminis ~]# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination [root@geminis ~]# [root@aries cluster]# iptables -L Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination [root@aries cluster]# Also the FS is mounted: [root@geminis ~]# df -h FilesystemSize Used Avail Use% Mounted on /dev/mapper/vg_geminis-lv_root 50G 4.2G 43G 9% / tmpfs 127G 80K 127G 1% /dev/shm /dev/sda1 477M 61M 391M 14% /boot /dev/mapper/vg_geminis-lv_home 36G 49M 34G 1% /home /dev/sdb1 3.7T 5.1G 3.7T 1% /var/lib/ceph/osd/ceph-43 [root@geminis ~]# [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. On Mar 8, 2015, at 2:08 PM, Thomas Foster thomas.foste...@gmail.commailto:thomas.foste...@gmail.com wrote: Have you checked firewall or made sure you created the mount for the osd correctly? Did you check the osd.43 log? On Mar 8, 2015 3:58 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Hi all, I had issues with a stuck PGs so I decided to star over and delete 1 OSDs host that was causing the issues I remove everything also purge data… Now I tried to active osd but still remains down in ceph osd tree command, I tried also to remove from crushmap (compiling and decompiling etc) and activate again from the bottom and still not working =( -11 43.56 host sagitario 9 3.63osd.9 up 1 109 3.63osd.109 up 1 110 3.63osd.110 up 1 111 3.63osd.111 up 1 112 3.63osd.112 up 1 113 3.63osd.113 up 1 114 3.63osd.114 up 1 115 3.63osd.115 up 1 116 3.63osd.116 up 1 117 3.63osd.117 up 1 118 3.63osd.118 up 1 119 3.63osd.119 up 1 -5 3.63host geminis 43 3.63osd.43 down0 3 0 osd.3 down0 osd.3 remains down =( I don’t know what else should I do Regards image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006.gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] OSD Remains down
I was thinking about all that osd take them out from crush mapping and removing and if I try to get them back again they took the same osd ids, maybe in cache or in some place they didnt complete was removed or something, but thats why I also Edit the crushmap and inject again I dont want to make a fresh install again since they are 120 osds 10 hosts is a big work :( Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 8, 2015, at 2:08 PM, Thomas Foster thomas.foste...@gmail.commailto:thomas.foste...@gmail.com wrote: Have you checked firewall or made sure you created the mount for the osd correctly? Did you check the osd.43 log? On Mar 8, 2015 3:58 PM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Hi all, I had issues with a stuck PGs so I decided to star over and delete 1 OSDs host that was causing the issues I remove everything also purge data… Now I tried to active osd but still remains down in ceph osd tree command, I tried also to remove from crushmap (compiling and decompiling etc) and activate again from the bottom and still not working =( -11 43.56 host sagitario 9 3.63osd.9 up 1 109 3.63osd.109 up 1 110 3.63osd.110 up 1 111 3.63osd.111 up 1 112 3.63osd.112 up 1 113 3.63osd.113 up 1 114 3.63osd.114 up 1 115 3.63osd.115 up 1 116 3.63osd.116 up 1 117 3.63osd.117 up 1 118 3.63osd.118 up 1 119 3.63osd.119 up 1 -5 3.63host geminis 43 3.63osd.43 down0 3 0 osd.3 down0 osd.3 remains down =( I don’t know what else should I do Regards image005.png Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:%2B52%2055%205267%203146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ image006.gif Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] OSD Remains down
Hi all, I had issues with a stuck PGs so I decided to star over and delete 1 OSDs host that was causing the issues I remove everything also purge data… Now I tried to active osd but still remains down in ceph osd tree command, I tried also to remove from crushmap (compiling and decompiling etc) and activate again from the bottom and still not working =( -11 43.56 host sagitario 9 3.63osd.9 up 1 109 3.63osd.109 up 1 110 3.63osd.110 up 1 111 3.63osd.111 up 1 112 3.63osd.112 up 1 113 3.63osd.113 up 1 114 3.63osd.114 up 1 115 3.63osd.115 up 1 116 3.63osd.116 up 1 117 3.63osd.117 up 1 118 3.63osd.118 up 1 119 3.63osd.119 up 1 -5 3.63host geminis 43 3.63osd.43 down0 3 0 osd.3 down0 osd.3 remains down =( I don’t know what else should I do Regards [cid:image005.png@01D00809.A6D502D0] Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 Cisco.comhttp://www.cisco.com/ [cid:image006.gif@01D00809.A6D502D0] Think before you print. This email may contain confidential and privileged material for the sole use of the intended recipient. Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the intended recipient (or authorized to receive for the recipient), please contact the sender by reply email and delete all copies of this message. Please click herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for Company Registration Information. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Permanente Mount RBD blocs device RHEL7
Sorry I couldn’t answer back sooner, I made it with
/etc/systemd/system/rbd-{ceph_pool}-{ceph_image}.service, I had some mistakes
in the name of the config file:
[Unit]
Description=RADOS block device mapping for {ceph_pool}/{ceph_image}
Conflicts=shutdown.target
Wants=network-online.target
After=NetworkManager-wait-online.service
[Service]
Type=oneshot
ExecStart=/sbin/modprobe rbd
ExecStart=/bin/sh -c /bin/echo {ceph_mon_ip}
name={ceph_admin},secret={ceph_key} {ceph_pool} {ceph_image} /sys/bus/rbd/add
TimeoutSec=0
RemainAfterExit=yes
[Install]
WantedBy=multi-user.target
Yes I did use systemctl and normal mount points in ftstab now is working =)
Thank you so mucho for all your help, Id like to take this message to ask you
the last thing that is concerned to me, I did a failure test shutting down 1
server, but while the server is down I cannot write in the filesystem, is that
a normal behaviour? I checked the crush algorithm policy and it has “host”
taken for the replica that means that each replica is going to be in different
hosts…
Thank you everyone!
[cid:image005.png@01D00809.A6D502D0]
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
Cisco.comhttp://www.cisco.com/
[cid:image006.gif@01D00809.A6D502D0]
Think before you print.
This email may contain confidential and privileged material for the sole use of
the intended recipient. Any review, use, distribution or disclosure by others
is strictly prohibited. If you are not the intended recipient (or authorized to
receive for the recipient), please contact the sender by reply email and delete
all copies of this message.
Please click
herehttp://www.cisco.com/web/about/doing_business/legal/cri/index.html for
Company Registration Information.
On Mar 7, 2015, at 3:15 AM, Gian
g...@pinguozzo.commailto:g...@pinguozzo.com wrote:
Hi,
are you using /etc/ceph/rbdmount as a 'mapping fstab' plus your mountpoints in
normal fstab plus the systemctl service ?
Gian
On 07 Mar 2015, at 05:26, Jesus Chavez (jeschave)
jesch...@cisco.commailto:jesch...@cisco.com wrote:
Still not working does anybody know show to automap and Mount rbd image on
redhat?
Regards
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146tel:+52%2055%205267%203146
Mobile: +51 1 5538883255tel:+51%201%205538883255
CCIE - 44433
On Mar 2, 2015, at 4:52 AM, Jesus Chavez (jeschave)
jesch...@cisco.commailto:jesch...@cisco.com wrote:
Thank you so much Alexandre! :)
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone: +52 55 5267 3146tel:+52%2055%205267%203146
Mobile: +51 1 5538883255tel:+51%201%205538883255
CCIE - 44433
On Mar 2, 2015, at 4:26 AM, Alexandre DERUMIER
aderum...@odiso.commailto:aderum...@odiso.com wrote:
Hi,
maybe this can help you:
http://www.sebastien-han.fr/blog/2013/11/22/map-slash-unmap-rbd-device-on-boot-slash-shutdown/
Regards,
Alexandre
- Mail original -
De: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com
À: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
Envoyé: Lundi 2 Mars 2015 11:14:49
Objet: [ceph-users] Permanente Mount RBD blocs device RHEL7
Hi all! I have been trying to get permanent my fs maked by the rbd device
mapping on rhel7 modifying /etc/fstab but everytime I reboot the server I lose
the mapping to the pool so the server gets stuck since It didnt find the
/dev/rbd0 device, does anybody know if there any procedure to not lose the
mapping or make the filesystem permanent?
Thanks!
Jesus Chavez
SYSTEMS ENGINEER-C.SALES
jesch...@cisco.commailto:jesch...@cisco.com
Phone : +52 55 5267 3146
Mobile: +51 1 5538883255
CCIE - 44433
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Permanente Mount RBD blocs device RHEL7
Still not working does anybody know show to automap and Mount rbd image on redhat? Regards Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 2, 2015, at 4:52 AM, Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com wrote: Thank you so much Alexandre! :) Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone: +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 On Mar 2, 2015, at 4:26 AM, Alexandre DERUMIER aderum...@odiso.commailto:aderum...@odiso.com wrote: Hi, maybe this can help you: http://www.sebastien-han.fr/blog/2013/11/22/map-slash-unmap-rbd-device-on-boot-slash-shutdown/ Regards, Alexandre - Mail original - De: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com À: ceph-users ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com Envoyé: Lundi 2 Mars 2015 11:14:49 Objet: [ceph-users] Permanente Mount RBD blocs device RHEL7 Hi all! I have been trying to get permanent my fs maked by the rbd device mapping on rhel7 modifying /etc/fstab but everytime I reboot the server I lose the mapping to the pool so the server gets stuck since It didnt find the /dev/rbd0 device, does anybody know if there any procedure to not lose the mapping or make the filesystem permanent? Thanks! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone : +52 55 5267 3146 Mobile: +51 1 5538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.commailto:ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Fwd: RPM Build Errors
someone in this DL had the thread error?
Checking for unpackaged file(s): /usr/lib/rpm/check-files
/home/vagrant/rpmbuild/BUILDROOT/calamari-server-1.3-rc_23_g4c41db3.el7.x86_64
Wrote:
/home/vagrant/rpmbuild/RPMS/x86_64/calamari-server-1.3-rc_23_g4c41db3.el7.x86_64.rpm
Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.TF02LW
--
ID: cp-artifacts-to-share calamari/repobuild/calamari-repo-*.tar.gz
Function: cmd.run
Name: cp calamari/repobuild/calamari-repo-*.tar.gz /git
Result: False
Comment: Command cp calamari/repobuild/calamari-repo-*.tar.gz /git run
Started: 18:38:22.222920
Duration: 12.591 ms
Changes:
--
pid:
855
retcode:
1
stderr:
cp: cannot stat 'calamari/repobuild/calamari-repo-*.tar.gz':
No such file or directory
stdout:
Begin forwarded message:
From: Jesus Chavez (jeschave) jesch...@cisco.commailto:jesch...@cisco.com
Subject: RPM Build Errors
Date: March 3, 2015 at 5:47:14 PM CST
To: ceph-calam...@ceph.commailto:ceph-calam...@ceph.com
Hi everyone I am having exactly the same issue, does anybody knew whats going
on on this?
Thanks!
I've seen this kind of compiler error with too little memory.
As for salt, it's set up by vagrant because it's using the salt
provider. What error are you seeing?
On Jan 5, 2015 4:46 AM, John Spray john.spray at
redhat.comhttp://lists.ceph.com/listinfo.cgi/ceph-calamari-ceph.com wrote:
Forwarding from ceph-users to ceph-calamari.
-- Forwarded message --
From: Tony unixfly at
gmail.comhttp://lists.ceph.com/listinfo.cgi/ceph-calamari-ceph.com
Date: Wed, Dec 24, 2014 at 7:03 PM
Subject: [ceph-users] Calamari
To: ceph-users at
ceph.comhttp://lists.ceph.com/listinfo.cgi/ceph-calamari-ceph.com
Has anyone else ran into this error? I've tried different versions of
GCC and even CentOS and RHEL to compile the calamari but continues to
fail and by the way the instructions on the ceph website are not
correct because the virtual used with vagrant isn't complete with
which ever versions they used to compile with and salt doesn't exist
on the virtual.
Here is the error message I'm having below:
I thought this was a memory issue but I changed memory in the virtual
and even cores from 1 to 4, 8, 16 without luck.
The below error still looks like a memory issue but I gave it 12G and
still failed.
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC
-fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64
-mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC
-I/usr/include/python2.6 -c
/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/build/cython/Cython/Compiler/Parsing.c
-o
build/temp.linux-x86_64-2.6/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/build/cython/Cython/Compiler/Parsing.o
{standard input}: Assembler messages:
{standard input}:19186: Warning: end of file not at
end of a line; newline inserted
{standard input}:19533: Error: unknown pseudo-op: `.lc2'
gcc: Internal error: Killed (program cc1)
Please submit a full bug report.
See http://bugzilla.redhat.com/bugzilla for
instructions.
error: command 'gcc' failed with exit status 1
Can't roll back Cython; was not uninstalled
Cleaning up...
Command
/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/bin/python -c
import
setuptools;__file__='/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/build/cython/setup.py';exec(compile(open(__file__).read().replace('\r\n',
'\n'), __file__, 'exec')) install --record
/tmp/pip-6R8jlS-record/install-record.txt
--single-version-externally-managed --install-headers
/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/include/site/python2.6
failed with error code 1 in
/home/vagrant/rpmbuild/BUILD/calamari-server-1.2.1/venv/build/cython
Storing complete log in /home/vagrant/.pip/pip.log
RPM build errors:
--
ID: cp-artifacts-to-share
calamari/repobuild/calamari-repo-rhel6.tar.gz
Function: cmd.run
Name: cp calamari/repobuild/calamari-repo-rhel6.tar.gz /git/
Result: True
Comment: Command cp
calamari/repobuild/calamari-repo-rhel6.tar.gz /git/ run
Started: 18:54:40.702746
Duration: 123.319 ms
Changes:
--
pid:
10380
retcode:
0
[ceph-users] Permanente Mount RBD blocs device RHEL7
Hi all! I have been trying to get permanent my fs maked by the rbd device mapping on rhel7 modifying /etc/fstab but everytime I reboot the server I lose the mapping to the pool so the server gets stuck since It didnt find the /dev/rbd0 device, does anybody know if there any procedure to not lose the mapping or make the filesystem permanent? Thanks! Jesus Chavez SYSTEMS ENGINEER-C.SALES jesch...@cisco.commailto:jesch...@cisco.com Phone : +52 55 5267 3146tel:+52%2055%205267%203146 Mobile: +51 1 5538883255tel:+51%201%205538883255 CCIE - 44433 ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
