Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

On 05/01/2018 01:43 PM, Oliver Freyermuth wrote: Hi all, Am 17.04.2018 um 19:38 schrieb Oliver Freyermuth: Am 17.04.2018 um 19:35 schrieb Daniel Gryniewicz: On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: Am 17.04.2018 um 17:34 schrieb Paul Emmerich: [...] We are right now using

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Hi all, Am 17.04.2018 um 19:38 schrieb Oliver Freyermuth: > Am 17.04.2018 um 19:35 schrieb Daniel Gryniewicz: >> On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: >>> Am 17.04.2018 um 17:34 schrieb Paul Emmerich: >> >>> [...] We are right now using the packages from

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Am 17.04.2018 um 19:35 schrieb Daniel Gryniewicz: > On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: >> Am 17.04.2018 um 17:34 schrieb Paul Emmerich: > >> [...] >>> >>> We are right now using the packages from >>> https://eu.ceph.com/nfs-ganesha/ since

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

On 04/17/2018 11:40 AM, Oliver Freyermuth wrote: Am 17.04.2018 um 17:34 schrieb Paul Emmerich: [...] We are right now using the packages from https://eu.ceph.com/nfs-ganesha/ since we would like not to have to build NFS Ganesha against Ceph

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Am 17.04.2018 um 17:34 schrieb Paul Emmerich: > > > 2018-04-16 18:24 GMT+02:00 Oliver Freyermuth >: > > Hi Paul, > > Am 16.04.2018 um 17:51 schrieb Paul Emmerich: > > Hi, > > > > can you try to get a

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

2018-04-16 18:24 GMT+02:00 Oliver Freyermuth : > Hi Paul, > > Am 16.04.2018 um 17:51 schrieb Paul Emmerich: > > Hi, > > > > can you try to get a stack trace from ganesha (with gdb or from procfs) > when it's stuck? > > I can try, as soon as it happens again. The

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Hi Paul, Am 16.04.2018 um 17:51 schrieb Paul Emmerich: > Hi, > > can you try to get a stack trace from ganesha (with gdb or from procfs) when > it's stuck? I can try, as soon as it happens again. The problem is that it's not fully stuck - only the other clients are stuck when trying to access

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Hi, can you try to get a stack trace from ganesha (with gdb or from procfs) when it's stuck? Also, try to upgrade to ganesha 2.6. I'm running a bigger deployment with ~30 ganesha 2.6 gateways that are quite stable so far. Paul 2018-04-16 17:30 GMT+02:00 Oliver Freyermuth

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Am 16.04.2018 um 08:58 schrieb Oliver Freyermuth: > Am 16.04.2018 um 02:43 schrieb Oliver Freyermuth: >> Am 15.04.2018 um 23:04 schrieb John Spray: >>> On Fri, Apr 13, 2018 at 5:16 PM, Oliver Freyermuth >>> wrote: Dear Cephalopodians, in our cluster

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Am 16.04.2018 um 02:43 schrieb Oliver Freyermuth: > Am 15.04.2018 um 23:04 schrieb John Spray: >> On Fri, Apr 13, 2018 at 5:16 PM, Oliver Freyermuth >> wrote: >>> Dear Cephalopodians, >>> >>> in our cluster (CentOS 7.4, EC Pool, Snappy compression, Luminous 12.2.4),

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Am 15.04.2018 um 23:04 schrieb John Spray: > On Fri, Apr 13, 2018 at 5:16 PM, Oliver Freyermuth > wrote: >> Dear Cephalopodians, >> >> in our cluster (CentOS 7.4, EC Pool, Snappy compression, Luminous 12.2.4), >> we often have all (~40) clients accessing one file in

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

On Fri, Apr 13, 2018 at 5:16 PM, Oliver Freyermuth wrote: > Dear Cephalopodians, > > in our cluster (CentOS 7.4, EC Pool, Snappy compression, Luminous 12.2.4), > we often have all (~40) clients accessing one file in readonly mode, even > with multiple processes per

Re: [ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Dear Cephalopodians, a small addition. As far as I know, the I/O the user is performing is based on the following directory structure: datafolder/some_older_tarball.tar.gz datafolder/sometarball.tar.gz datafolder/processing_number_2/ datafolder/processing_number_3/

[ceph-users] CephFS MDS stuck (failed to rdlock when getattr / lookup)

Dear Cephalopodians, in our cluster (CentOS 7.4, EC Pool, Snappy compression, Luminous 12.2.4), we often have all (~40) clients accessing one file in readonly mode, even with multiple processes per client doing that. Sometimes (I do not yet know when, nor why!) the MDS ends up in a situation