Re: [ceph-users] redundancy with 2 nodes
Did you shut down the node with 2 mon?
I think it might be impossible to have redundancy with only 2 node, paxos
quorum is the reason:
Say you have N (N=2K+1) monitors, you always have a node(let's named it node A)
with majority number of MONs(>= K+1), another node(node B) with minority number
of MONs(<=K)
To form a quorum ,you need at least K+1 alive MONs, so, if node B down,
everything is good. But if node A goes down, you can never have a majority with
<=K monitor.
From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of Jiri
Kanicky
Sent: Thursday, January 1, 2015 12:50 PM
To: ceph-users@lists.ceph.com
Subject: Re: [ceph-users] redundancy with 2 nodes
Hi,
I noticed this message after shutting down the other node. You might be right
that I need 3 monitors.
2015-01-01 15:47:35.990260 7f22858dd700 0 monclient: hunting for new mon
But what is quite unexpected is that you cannot run even "ceph status" on the
running node t find out the state of the cluster.
Thx Jiri
On 1/01/2015 15:46, Jiri Kanicky wrote:
Hi,
I have:
- 2 monitors, one on each node
- 4 OSDs, two on each node
- 2 MDS, one on each node
Yes, all pools are set with size=2 and min_size=1
cephadmin@ceph1:~$ ceph osd dump
epoch 88
fsid bce2ff4d-e03b-4b75-9b17-8a48ee4d7788
created 2014-12-27 23:38:00.455097
modified 2014-12-30 20:45:51.343217
flags
pool 0 'rbd' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins
p g_num 256 pgp_num 256 last_change 86 flags hashpspool
stripe_width 0
pool 1 'media' replicated size 2 min_size 1 crush_ruleset 0 object_hash
rjenkins pg_num 256 pgp_num 256 last_change 60 flags
hashpspool stripe_width 0
pool 2 'data' replicated size 2 min_size 1 crush_ruleset 0 object_hash rjenkins
pg_num 256 pgp_num 256 last_change 63 flags hashpspool
stripe_width 0
pool 3 'cephfs_test' replicated size 2 min_size 1 crush_ruleset 0 object_hash
rj enkins pg_num 256 pgp_num 256 last_change 71 flags
hashpspool crash_replay_inter val 45 stripe_width 0
pool 4 'cephfs_metadata' replicated size 2 min_size 1 crush_ruleset 0
object_has h rjenkins pg_num 256 pgp_num 256 last_change 69
flags hashpspool stripe_width 0
max_osd 4
osd.0 up in weight 1 up_from 55 up_thru 86 down_at 51 last_clean_interval
[39 ,50) 192.168.30.21:6800/17319 10.1.1.21:6800/17319
10.1.1.21:6801/17319 192.168. 30.21:6801/17319 exists,up
4f3172e1-adb8-4ca3-94af-6f0b8fcce35a
osd.1 up in weight 1 up_from 57 up_thru 86 down_at 53 last_clean_interval
[41 ,52) 192.168.30.21:6803/17684 10.1.1.21:6802/17684
10.1.1.21:6804/17684 192.168. 30.21:6805/17684 exists,up
1790347a-94fa-4b81-b429-1e7c7f11d3fd
osd.2 up in weight 1 up_from 79 up_thru 86 down_at 74 last_clean_interval
[13 ,73) 192.168.30.22:6801/3178 10.1.1.22:6800/3178
10.1.1.22:6801/3178 192.168.30. 22:6802/3178 exists,up
5520835f-c411-4750-974b-34e9aea2585d
osd.3 up in weight 1 up_from 81 up_thru 86 down_at 72 last_clean_interval
[20 ,71) 192.168.30.22:6804/3414 10.1.1.22:6802/3414
10.1.1.22:6803/3414 192.168.30. 22:6805/3414 exists,up
25e62059-6392-4a69-99c9-214ae335004
Thx Jiri
On 1/01/2015 15:21, Lindsay Mathieson wrote:
On Thu, 1 Jan 2015 02:59:05 PM Jiri Kanicky wrote:
I would expect that if I shut down one node, the system will keep
running. But when I tested it, I cannot even execute "ceph status"
command on the running node.
2 osd Nodes, 3 Mon nodes here, works perfectly for me.
How many monitors do you have?
Maybe you need a third monitor only node for quorum?
I set "osd_pool_default_size = 2" (min_size=1) on all pools, so I
thought that each copy will reside on each node. Which means that if 1
node goes down the second one will be still operational.
does:
ceph osd pool get {pool name} size
return 2
ceph osd pool get {pool name} min_size
return 1
___
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com<mailto:ceph-users@lists.ceph.com>
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
On 01/01/15 23:16, Christian Balzer wrote:
Hello,
On Thu, 01 Jan 2015 18:25:47 +1300 Mark Kirkwood wrote:
but I agree that you should probably not get a HEALTH OK status when you
have just setup 2 (or in fact any even number of) monitors...HEALTH WARN
would make more sense, with a wee message suggesting adding at least one
more!
I think what Jiri meant is that wen the whole cluster goes into a deadlock
due to loosing monitor quorum, ceph -s etc won't work anymore either.
Right - but looking at health output from his earlier post:
cephadmin@ceph1:~$ ceph status
cluster bce2ff4d-e03b-4b75-9b17-8a48ee4d7788
health HEALTH_OK
monmap e1: 2 mons at
{ceph1=192.168.30.21:6789/0,ceph2=192.168.30.22:6789/0}, election epoch
12, quorum 0,1 ceph1,ceph2
mdsmap e7: 1/1/1 up {0=ceph1=up:active}, 1 up:standby
osdmap e88: 4 osds: 4 up, 4 in
pgmap v2051: 1280 pgs, 5 pools, 13184 MB data, 3328 objects
26457 MB used, 11128 GB / 11158 GB avail
1280 active+clean
...if he had received some sort of caution about the number of mons
instead of HEALTH OK from that health status, then he might have added
another *before* everything locked up. That's what I was meaning before.
And while the cluster rightfully shouldn't be doing anything in such a
state, querying the surviving/reachable monitor and being told as much
would indeed be a nice feature, as opposed to deafening silence.
Sure, getting nothing is highly undesirable.
As for your suggestion, while certainly helpful it is my not so humble
opinion than the the WARN state right now is totally overloaded and quite
frankly bogus.
This is particularly a problem with monitor plugins that just pick up the
WARN state without further discrimination.
Yeah, I agree that WARN is hopelessly overloaded. In the past I have to
dig backward in the logs to see what the warning is actually about, and
if it is really something that needs attention.
regards
Mark
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
Hi,
I noticed this message after shutting down the other node. You might be
right that I need 3 monitors.
2015-01-01 15:47:35.990260 7f22858dd700 0 monclient: hunting for new mon
But what is quite unexpected is that you cannot run even "ceph status"
on the running node t find out the state of the cluster.
Thx Jiri
On 1/01/2015 15:46, Jiri Kanicky wrote:
Hi,
I have:
- 2 monitors, one on each node
- 4 OSDs, two on each node
- 2 MDS, one on each node
Yes, all pools are set with size=2 and min_size=1
cephadmin@ceph1:~$ ceph osd dump
epoch 88
fsid bce2ff4d-e03b-4b75-9b17-8a48ee4d7788
created 2014-12-27 23:38:00.455097
modified 2014-12-30 20:45:51.343217
flags
pool 0 'rbd' replicated *size 2 min_size 1* crush_ruleset 0
object_hash rjenkins p g_num 256 pgp_num 256
last_change 86 flags hashpspool stripe_width 0
pool 1 'media' replicated *size 2 min_size 1* crush_ruleset 0
object_hash rjenkins pg_num 256 pgp_num 256
last_change 60 flags hashpspool stripe_width 0
pool 2 'data' replicated size *2 min_size 1* crush_ruleset 0
object_hash rjenkins pg_num 256 pgp_num 256
last_change 63 flags hashpspool stripe_width 0
pool 3 'cephfs_test' replicated *size 2 min_size 1* crush_ruleset 0
object_hash rj enkins pg_num 256 pgp_num 256
last_change 71 flags hashpspool crash_replay_inter
val 45 stripe_width 0
pool 4 'cephfs_metadata' replicated *size 2 min_size 1* crush_ruleset
0 object_has h rjenkins pg_num 256 pgp_num 256
last_change 69 flags hashpspool stripe_width 0
max_osd 4
osd.0 up in weight 1 up_from 55 up_thru 86 down_at 51
last_clean_interval [39 ,50) 192.168.30.21:6800/17319
10.1.1.21:6800/17319 10.1.1.21:6801/17319 192.168.
30.21:6801/17319 exists,up 4f3172e1-adb8-4ca3-94af-6f0b8fcce35a
osd.1 up in weight 1 up_from 57 up_thru 86 down_at 53
last_clean_interval [41 ,52) 192.168.30.21:6803/17684
10.1.1.21:6802/17684 10.1.1.21:6804/17684 192.168.
30.21:6805/17684 exists,up 1790347a-94fa-4b81-b429-1e7c7f11d3fd
osd.2 up in weight 1 up_from 79 up_thru 86 down_at 74
last_clean_interval [13 ,73) 192.168.30.22:6801/3178
10.1.1.22:6800/3178 10.1.1.22:6801/3178 192.168.30.
22:6802/3178 exists,up 5520835f-c411-4750-974b-34e9aea2585d
osd.3 up in weight 1 up_from 81 up_thru 86 down_at 72
last_clean_interval [20 ,71) 192.168.30.22:6804/3414
10.1.1.22:6802/3414 10.1.1.22:6803/3414 192.168.30.
22:6805/3414 exists,up 25e62059-6392-4a69-99c9-214ae335004
Thx Jiri
On 1/01/2015 15:21, Lindsay Mathieson wrote:
On Thu, 1 Jan 2015 02:59:05 PM Jiri Kanicky wrote:
I would expect that if I shut down one node, the system will keep
running. But when I tested it, I cannot even execute "ceph status"
command on the running node.
2 osd Nodes, 3 Mon nodes here, works perfectly for me.
How many monitors do you have?
Maybe you need a third monitor only node for quorum?
I set "osd_pool_default_size = 2" (min_size=1) on all pools, so I
thought that each copy will reside on each node. Which means that if 1
node goes down the second one will be still operational.
does:
ceph osd pool get {pool name} size
return 2
ceph osd pool get {pool name} min_size
return 1
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
Hello, On Thu, 01 Jan 2015 18:25:47 +1300 Mark Kirkwood wrote: > The number of monitors recommended and the fact that a voting quorum is > the way it works is covered here: > > http://ceph.com/docs/master/rados/deployment/ceph-deploy-mon/ > > but I agree that you should probably not get a HEALTH OK status when you > have just setup 2 (or in fact any even number of) monitors...HEALTH WARN > would make more sense, with a wee message suggesting adding at least one > more! > I think what Jiri meant is that wen the whole cluster goes into a deadlock due to loosing monitor quorum, ceph -s etc won't work anymore either. And while the cluster rightfully shouldn't be doing anything in such a state, querying the surviving/reachable monitor and being told as much would indeed be a nice feature, as opposed to deafening silence. As for your suggestion, while certainly helpful it is my not so humble opinion than the the WARN state right now is totally overloaded and quite frankly bogus. This is particularly a problem with monitor plugins that just pick up the WARN state without further discrimination. And some WARN states like slow requests are pretty much an ERR state for most people, stalled requests for more than 30 seconds (or days!) are a sign of something massively wrong and likely to have customer/client impact. I think a neat solution would be the ability to assign all possible problem states a value like ERR, WARN, NOTE. A cluster with just 1 or 2 monitors or having scrub disabled is (for me) worth a NOTE, but not a WARN. Christian > Regards > > Mark > > > On 01/01/15 18:06, Jiri Kanicky wrote: > > Hi, > > > > I think you are right. I was too focused on the following line in docs: > > "A cluster will run fine with a single monitor; however,*a single > > monitor is a single-point-of-failure*." I will try to add another > > monitor. Hopefully, this will fix my issue. > > > > Anyway, I think that "ceph status" or "ceph health" should report at > > least something in such state. Its quite weird that everything stops... > > > > Thank you > > Jiri > > > > On 1/01/2015 15:51, Lindsay Mathieson wrote: > >> On Thu, 1 Jan 2015 03:46:33 PM Jiri Kanicky wrote: > >>> Hi, > >>> > >>> I have: > >>> - 2 monitors, one on each node > >>> - 4 OSDs, two on each node > >>> - 2 MDS, one on each node > >> POOMA U here, but I don't think you can reach quorum with one out of > >> two monitors, you need a odd number: > >> > >> http://ceph.com/docs/master/rados/configuration/mon-config-ref/#monitor-quorum > >> > >> Perhaps try removing one monitor, so you only have one left, then > >> take the node without a monitor down. > >> > >> > >> > >> ___ > >> ceph-users mailing list > >> ceph-users@lists.ceph.com > >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > > > > > ___ > > ceph-users mailing list > > ceph-users@lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > > > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/ ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
The number of monitors recommended and the fact that a voting quorum is the way it works is covered here: http://ceph.com/docs/master/rados/deployment/ceph-deploy-mon/ but I agree that you should probably not get a HEALTH OK status when you have just setup 2 (or in fact any even number of) monitors...HEALTH WARN would make more sense, with a wee message suggesting adding at least one more! Regards Mark On 01/01/15 18:06, Jiri Kanicky wrote: Hi, I think you are right. I was too focused on the following line in docs: "A cluster will run fine with a single monitor; however,*a single monitor is a single-point-of-failure*." I will try to add another monitor. Hopefully, this will fix my issue. Anyway, I think that "ceph status" or "ceph health" should report at least something in such state. Its quite weird that everything stops... Thank you Jiri On 1/01/2015 15:51, Lindsay Mathieson wrote: On Thu, 1 Jan 2015 03:46:33 PM Jiri Kanicky wrote: Hi, I have: - 2 monitors, one on each node - 4 OSDs, two on each node - 2 MDS, one on each node POOMA U here, but I don't think you can reach quorum with one out of two monitors, you need a odd number: http://ceph.com/docs/master/rados/configuration/mon-config-ref/#monitor-quorum Perhaps try removing one monitor, so you only have one left, then take the node without a monitor down. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
Hi, I think you are right. I was too focused on the following line in docs: "A cluster will run fine with a single monitor; however,*a single monitor is a single-point-of-failure*." I will try to add another monitor. Hopefully, this will fix my issue. Anyway, I think that "ceph status" or "ceph health" should report at least something in such state. Its quite weird that everything stops... Thank you Jiri On 1/01/2015 15:51, Lindsay Mathieson wrote: On Thu, 1 Jan 2015 03:46:33 PM Jiri Kanicky wrote: Hi, I have: - 2 monitors, one on each node - 4 OSDs, two on each node - 2 MDS, one on each node POOMA U here, but I don't think you can reach quorum with one out of two monitors, you need a odd number: http://ceph.com/docs/master/rados/configuration/mon-config-ref/#monitor-quorum Perhaps try removing one monitor, so you only have one left, then take the node without a monitor down. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
On Thu, 01 Jan 2015 14:59:05 +1100 Jiri Kanicky wrote:
>
> monmap e1: 2 mons at
> {ceph1=192.168.30.21:6789/0,ceph2=192.168.30.22:6789/0}, election epoch
> 12, quorum 0,1 ceph1,ceph2
>
That's your problem, re-read the Ceph documentation about Paxos.
You need a third monitor to retain a viable quorum if one node goes down.
That is also why the next useful increase of monitors is from 3 to 5, so
you can loose 2 at a time.
Christian
--
Christian BalzerNetwork/Systems Engineer
ch...@gol.com Global OnLine Japan/Fusion Communications
http://www.gol.com/
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
On Thu, 1 Jan 2015 03:46:33 PM Jiri Kanicky wrote: > Hi, > > I have: > - 2 monitors, one on each node > - 4 OSDs, two on each node > - 2 MDS, one on each node POOMA U here, but I don't think you can reach quorum with one out of two monitors, you need a odd number: http://ceph.com/docs/master/rados/configuration/mon-config-ref/#monitor-quorum Perhaps try removing one monitor, so you only have one left, then take the node without a monitor down. -- Lindsay signature.asc Description: This is a digitally signed message part. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
Hi,
I have:
- 2 monitors, one on each node
- 4 OSDs, two on each node
- 2 MDS, one on each node
Yes, all pools are set with size=2 and min_size=1
cephadmin@ceph1:~$ ceph osd dump
epoch 88
fsid bce2ff4d-e03b-4b75-9b17-8a48ee4d7788
created 2014-12-27 23:38:00.455097
modified 2014-12-30 20:45:51.343217
flags
pool 0 'rbd' replicated *size 2 min_size 1* crush_ruleset 0 object_hash
rjenkins p g_num 256 pgp_num 256 last_change 86 flags
hashpspool stripe_width 0
pool 1 'media' replicated *size 2 min_size 1* crush_ruleset 0
object_hash rjenkins pg_num 256 pgp_num 256
last_change 60 flags hashpspool stripe_width 0
pool 2 'data' replicated size *2 min_size 1* crush_ruleset 0 object_hash
rjenkins pg_num 256 pgp_num 256 last_change 63 flags
hashpspool stripe_width 0
pool 3 'cephfs_test' replicated *size 2 min_size 1* crush_ruleset 0
object_hash rj enkins pg_num 256 pgp_num 256
last_change 71 flags hashpspool crash_replay_inter val
45 stripe_width 0
pool 4 'cephfs_metadata' replicated *size 2 min_size 1* crush_ruleset 0
object_has h rjenkins pg_num 256 pgp_num 256
last_change 69 flags hashpspool stripe_width 0
max_osd 4
osd.0 up in weight 1 up_from 55 up_thru 86 down_at 51
last_clean_interval [39 ,50) 192.168.30.21:6800/17319
10.1.1.21:6800/17319 10.1.1.21:6801/17319 192.168.
30.21:6801/17319 exists,up 4f3172e1-adb8-4ca3-94af-6f0b8fcce35a
osd.1 up in weight 1 up_from 57 up_thru 86 down_at 53
last_clean_interval [41 ,52) 192.168.30.21:6803/17684
10.1.1.21:6802/17684 10.1.1.21:6804/17684 192.168.
30.21:6805/17684 exists,up 1790347a-94fa-4b81-b429-1e7c7f11d3fd
osd.2 up in weight 1 up_from 79 up_thru 86 down_at 74
last_clean_interval [13 ,73) 192.168.30.22:6801/3178
10.1.1.22:6800/3178 10.1.1.22:6801/3178 192.168.30.
22:6802/3178 exists,up 5520835f-c411-4750-974b-34e9aea2585d
osd.3 up in weight 1 up_from 81 up_thru 86 down_at 72
last_clean_interval [20 ,71) 192.168.30.22:6804/3414
10.1.1.22:6802/3414 10.1.1.22:6803/3414 192.168.30.
22:6805/3414 exists,up 25e62059-6392-4a69-99c9-214ae335004
Thx Jiri
On 1/01/2015 15:21, Lindsay Mathieson wrote:
On Thu, 1 Jan 2015 02:59:05 PM Jiri Kanicky wrote:
I would expect that if I shut down one node, the system will keep
running. But when I tested it, I cannot even execute "ceph status"
command on the running node.
2 osd Nodes, 3 Mon nodes here, works perfectly for me.
How many monitors do you have?
Maybe you need a third monitor only node for quorum?
I set "osd_pool_default_size = 2" (min_size=1) on all pools, so I
thought that each copy will reside on each node. Which means that if 1
node goes down the second one will be still operational.
does:
ceph osd pool get {pool name} size
return 2
ceph osd pool get {pool name} min_size
return 1
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] redundancy with 2 nodes
On Thu, 1 Jan 2015 02:59:05 PM Jiri Kanicky wrote:
> I would expect that if I shut down one node, the system will keep
> running. But when I tested it, I cannot even execute "ceph status"
> command on the running node.
2 osd Nodes, 3 Mon nodes here, works perfectly for me.
How many monitors do you have?
Maybe you need a third monitor only node for quorum?
>
> I set "osd_pool_default_size = 2" (min_size=1) on all pools, so I
> thought that each copy will reside on each node. Which means that if 1
> node goes down the second one will be still operational.
does:
ceph osd pool get {pool name} size
return 2
ceph osd pool get {pool name} min_size
return 1
--
Lindsay
signature.asc
Description: This is a digitally signed message part.
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] redundancy with 2 nodes
Hi,
Is it possible to achieve redundancy with 2 nodes only?
cephadmin@ceph1:~$ ceph osd tree
# idweight type name up/down reweight
-1 10.88 root default
-2 5.44host ceph1
0 2.72osd.0 up 1
1 2.72osd.1 up 1
-3 5.44host ceph2
2 2.72osd.2 up 1
3 2.72osd.3 up 1
cephadmin@ceph1:~$ ceph status
cluster bce2ff4d-e03b-4b75-9b17-8a48ee4d7788
health HEALTH_OK
monmap e1: 2 mons at
{ceph1=192.168.30.21:6789/0,ceph2=192.168.30.22:6789/0}, election epoch
12, quorum 0,1 ceph1,ceph2
mdsmap e7: 1/1/1 up {0=ceph1=up:active}, 1 up:standby
osdmap e88: 4 osds: 4 up, 4 in
pgmap v2051: 1280 pgs, 5 pools, 13184 MB data, 3328 objects
26457 MB used, 11128 GB / 11158 GB avail
1280 active+clean
I would expect that if I shut down one node, the system will keep
running. But when I tested it, I cannot even execute "ceph status"
command on the running node.
I set "osd_pool_default_size = 2" (min_size=1) on all pools, so I
thought that each copy will reside on each node. Which means that if 1
node goes down the second one will be still operational.
I think my assumptions are wrong, but I could not find the explanation why.
Thanks Jiri
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
