I've installed a little, four node Ceph (0.47.2) cluster using Xen
virtual machines for testing, and when I run bonnie against a (kernel
driver) mount of it, it seems to be somewhat flaky (disturbing log
messages, occasional binary deaths), and very slow. I expected some
slowness, with all the vir
On Fri, Jun 29, 2012 at 11:07 AM, Gregory Farnum wrote:
>>> the "Designing a cluster guide"
>>> http://wiki.ceph.com/wiki/Designing_a_cluster is pretty good but it
>>> still leaves some questions unanswered.
Oh, thank you. I've been poking through the Ceph docs, but somehow
had not managed to tu
On Fri, Jun 29, 2012 at 11:55 AM, Gregory Farnum wrote:
> So right now you're using the Ceph filesystem, rather than RBD, right?
Right, CephFS. I'm actually not even very clear on what RBD is, and
how one might use it, but I'm sure I'll understand that in the
fullness of time. I came to Ceph fr
On Fri, Jun 29, 2012 at 11:50 AM, Gregory Farnum wrote:
> If you lose a journal, you lose the OSD.
Really? Everything? Not just recent commits? I would have hoped it
would just come back up in an old state. Replication should have
already been taking care of regaining redundancy for the stuff
On Fri, Jun 29, 2012 at 2:06 PM, Gregory Farnum wrote:
> Okay, there's two things I'd do here. First, create a cluster that
> only has one MDS — the multi-MDS system is significantly less stable.
Ok, will do.
> Second, you've got 3 monitors doing frequent fsyncs, and 4 OSDs doing
> frequent sync
On Fri, Jun 29, 2012 at 2:11 PM, Gregory Farnum wrote:
> Well, actually this depends on the filesystem you're using. With
> btrfs, the OSD will roll back to a consistent state, but you don't
> know how out-of-date that state is.
Ok, so assuming btrfs, then a single machine failure with a ramdisk
On Fri, Jun 29, 2012 at 2:14 PM, Brian Edmonds wrote:
> In any case, I'll rebuild the current four machines into one MON, one
> MDS and two OSDs and see how that goes. Thanks for the advice.
Just rebuilt as described, and it is significantly more stable.
Changing and listing dir
