[ceph-users] Cluster issue - pgs degraded, recovering, stale, etc.

Hello. I have a two node cluster with 4x replicas for all objects distributed between the two nodes (two copies on each node). I recently converted my OSDs from BTRFS to XFS (BTRFS was slow) by removing / preparing / activating OSDs on each node (one at at time) as XFS allowing cluster to

Re: [ceph-users] CRUSH Rule Review - Not replicating correctly

EGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Not that I know of. > - > Robert LeBlanc > PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 > > > On Mon, Jan 18, 2016 at 10:33 AM, deeepdish wrote: >> Thanks Robert. Will de

Re: [ceph-users] CRUSH Rule Review - Not replicating correctly

gt; VmUdnrdpfxzR5c8dmOELJnpNS6MTT56B8A4kKmqbbHCEKpZ83piG7uwqc+6f > RKkQ > =gp/0 > -END PGP SIGNATURE- > > Robert LeBlanc > PGP Fingerprint 79A2 9CA4 6CC4 45DD A904 C70E E654 3BB2 FA62 B9F1 > > > On Sun, Jan 17, 2016 at 6:31 PM, deeepdish <deeepd

[ceph-users] CRUSH Rule Review - Not replicating correctly

Hi Everyone, Looking for a double check of my logic and crush map.. Overview: - osdgroup bucket type defines failure domain within a host of 5 OSDs + 1 SSD. Therefore 5 OSDs (all utilizing the same journal) constitute an osdgroup bucket. Each host has 4 osdgroups. - 6 monitors - Two node

Re: [ceph-users] Help! OSD host failure - recovery without rebuilding OSDs

f the journal is linked to a /dev/sdx, make sure it's in the same spot as it > was before.. > > All the best of luck > /Josef > > On 25 Dec 2015 05:39, "deeepdish" <deeepd...@gmail.com > <mailto:deeepd...@gmail.com>> wrote: > Hello, > >

[ceph-users] Help! OSD host failure - recovery without rebuilding OSDs

Hello, Had an interesting issue today. My OSD hosts are booting off a USB key which, you guessed it has a root partition on there. All OSDs are mounted. My USB key failed on one of my OSD hosts, leaving the data on OSDs inaccessible to the rest of my cluster. I have multiple monitors

Re: [ceph-users] [SOLVED] Monitor rename / recreate issue -- probing state

. Syncing the key and redeploying monitors got them to peer and establish quorum. > On Dec 14, 2015, at 11:10 , deeepdish <deeepd...@gmail.com> wrote: > > Joao, > > Please see below. I think you’re totally right on: > >> I suspect they may already

Re: [ceph-users] Monitor rename / recreate issue -- probing state

On 12/10/2015 04:00 AM, deeepdish wrote: > Hello, > > I encountered a strange issue when rebuilding monitors reusing same > hostnames, however different IPs. > > Steps to reproduce: > > - Build monitor using ceph-deploy create mon > - Remove monitor > via http://d

Re: [ceph-users] Monitor rename / recreate issue -- probing state

t; }, { "rank": 2, "name": "smon02s", "addr": "0.0.0.0:0\/2" }, { "rank": 3, "name": "b02s08"

[ceph-users] Monitor rename / recreate issue -- probing state

Hello, I encountered a strange issue when rebuilding monitors reusing same hostnames, however different IPs. Steps to reproduce: - Build monitor using ceph-deploy create mon - Remove monitor via http://docs.ceph.com/docs/master/rados/operations/add-or-rm-mons/ (remove monitor) — I didn’t

[ceph-users] XFS calltrace exporting RBD via NFS

Hello, This is the second time I experienced this so I thought and post to get some perspective. When this first happened, I suspected the kernel, and upgraded from 3.18.22 to 3.18.23. Scenario: - lab scenario - single osd host — osdhost01. Supermicro X8DTE-F - 2x X5570 + 48G RAM + 20x

Re: [ceph-users] hanging nfsd requests on an RBD to NFS gateway

@John-Paul Robinson: I’ve also experienced nfs being blocked when serving rbd devices (XFS system). In my scenario I had rbd device mapped on an OSD host and nfs exported (lab scenario). Log entries below.. Running Centos 7 w/ 3.10.0-229.14.1.el7.x86_64. Next step for me is to compile

[ceph-users] Cache tier full not evicting

Hi Everyone, Getting close to cracking my understanding of cache tiering, and ec pools. Stuck on one anomaly which I do not understand — spent hours reviewing docs online, can’t seem to pin point what I’m doing wrong. Referencing http://ceph.com/docs/master/rados/operations/cache-tiering/

Re: [ceph-users] [SOLVED] Cache tier full not evicting

l size. > > From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com > <mailto:ceph-users-boun...@lists.ceph.com>] On Behalf Of deeepdish > Sent: 14 September 2015 16:27 > To: ceph-users@lists.ceph.com <mailto:ceph-users@lists.ceph.com> > Subject: [ceph-users

Re: [ceph-users] SOLVED: CRUSH odd bucket affinity / persistence

:47 , Nick Fisk <n...@fisk.me.uk> wrote: > >> -Original Message- >> From: ceph-users [mailto:ceph-users-boun...@lists.ceph.com] On Behalf Of >> deeepdish >> Sent: 13 September 2015 02:47 >> To: Johannes Formann <mlm...@formann.de> >> Cc:

[ceph-users] CRUSH odd bucket affinity / persistence

Hello, I’m having a (strange) issue with OSD bucket persistence / affinity on my test cluster.. The cluster is PoC / test, by no means production. Consists of a single OSD / MON host + another MON running on a KVM VM. Out of 12 OSDs I’m trying to get osd.10 and osd.11 to be part of the

Re: [ceph-users] CRUSH odd bucket affinity / persistence

Johannes, Thank you — "osd crush update on start = false” did the trick. I wasn’t aware that ceph has automatic placement logic for OSDs (http://permalink.gmane.org/gmane.comp.file-systems.ceph.user/9035 ). This

[ceph-users] EC + RBD Possible?

Hello. I wasn’t able to obtain a clear answer in my googling and reading official Ceph docs if Erasure Coded pools are possible/supported for RBD access? The idea is to have block (cold) storage for archival purposes. I would access an RBD device and format it as EXT or XFS for block use.

[ceph-users] Cache Tiering vs. OSD Journal

Hello. Quick question RE: cache tiering vs. OSD journals. As I understand it, SSD acceleration is possible at the pool or OSD level. When considering cache tiering, should I still put OSD journals on SSDs or should they be disabled altogether. Can a single SSD pool function as a

[ceph-users] ceph-deploy Errors - Fedora 21

Hello. I’m having an issue with ceph-deploy on Fedora 21. - Installed ceph-deploy via ‘yum install ceph-deploy' - created non-root user - assigned sudo privs as per documentation - http://ceph.com/docs/master/rados/deployment/preflight-checklist/

[ceph-users] ceph-deploy Errors - Fedora 21

Hello. I’m having an issue with ceph-deploy on Fedora 21. - Installed ceph-deploy via ‘yum install ceph-deploy' - created non-root user - assigned sudo privs as per documentation - http://ceph.com/docs/master/rados/deployment/preflight-checklist/