There are multiple disks per server, and will have one OSD for each disk. Is that okay?
Thanks again. From: Donny Davis [mailto:do...@fortnebula.com] Sent: Tuesday, April 03, 2018 10:12 AM To: ST Wong (ITSC) Cc: Ronny Aasen; ceph-users@lists.ceph.com Subject: Re: [ceph-users] split brain case Do you only have one OSD per server? Not that is really matters... because all of the above stated is true in any case. Just curious On Mon, Apr 2, 2018 at 6:40 PM, ST Wong (ITSC) <s...@itsc.cuhk.edu.hk<mailto:s...@itsc.cuhk.edu.hk>> wrote: Hi, >how many servers are your osd's split over ? keep in mind that ceph's default >picks one osd from each host. so you would need minimum 4 osd hosts in total >to be able to use 4+2 pools and with only 4 hosts you have no failuredomain. >but 4 hosts in the minimum sane starting point for a regular small cluster >with 3+2 pools (you can loose a node and ceph selfheals as long as there are >enough freespace. We’ll have 8 servers to split over (4 in each room). Thanks. Best Rgds, /st wong From: Ronny Aasen [mailto:ronny+ceph-us...@aasen.cx<mailto:ronny%2bceph-us...@aasen.cx>] Sent: Friday, March 30, 2018 3:18 AM To: ST Wong (ITSC); ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com> Subject: Re: [ceph-users] split brain case On 29.03.2018 11:13, ST Wong (ITSC) wrote: Hi, Thanks. > ofcourse the 4 osd's left working now want to selfheal by recreating all > objects stored on the 4 split off osd's and have a huge recovery job. and you > may risk that the osd's goes into too_full error, unless you have free space > in your osd's to recreate all the data in the defective part of the cluster. > or they will be stuck in recovery mode until you get the second room running, > this depends on your crush map. Means we’ve to made 4 OSD machines sufficient space to hold all data and thus the usable space will be halved? yes if you want to be able to be able to operatate one room as if it was the whole cluster (HA) then you need this. also if you want to have 4+2 instead of 3+2 pool size to avoid the blocking during recovery, that would take a whole lot of ekstra space. you can optionally let the cluster run degraded with 4+2 while one room is down. or temporary set pools to 2+2 while the other room is down, to reduce the space requirements. > point in that slitting the cluster hurts. and if HA is the most important > then you may want to check out rbd mirror. Will consider when there is budget to setup another ceph cluster for rdb mirror. i do not know your needs or applications, but while you only have 2 rooms you may just think of it as a single cluster that just happen to occupy 2 rooms. but with that few osd's you should perhaps just put the cluster in a single room the pain of splitting a cluster down the middle is quite significant. and i would perhaps use resources to improve the redundancy of the networks between the buildings instead. have multiple paths between the buildings to prevent service disruption in the building that does not house the cluster. having 5 mons is quite a lot. i think most clusters have 3 mons up into several hundred osd hosts how many servers are your osd's split over ? keep in mind that ceph's default picks one osd from each host. so you would need minimum 4 osd hosts in total to be able to use 4+2 pools and with only 4 hosts you have no failuredomain. but 4 hosts in the minimum sane starting point for a regular small cluster with 3+2 pools (you can loose a node and ceph selfheals as long as there are enough freespace. kind regards Ronny Aasen _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
