On Tue, Feb 2, 2016 at 10:09 PM, Michael Metz-Martini | SpeedPartner
GmbH <m...@speedpartner.de> wrote:
> Hi,
>
> we're experiencing some strange issues running ceph 0.87 in our, I
> think, quite large cluster (taking number of objects as a measurement).
>
> mdsmap e721086: 1/1/1 up {0=storagemds01=up:active}, 2 up:standby
> osdmap e143048: 92 osds: 92 up, 92 in
> flags noout,noscrub,nodeep-scrub
> pgmap v45790682: 4736 pgs, 6 pools, 109 TB data, 3841 Mobjects
> 255 TB used, 48892 GB / 303 TB avail
>
> Putting some higher load via cephfs on the cluster leads to messages
> like mds0: Client X failing to respond to capability release after some
> minutes. Requests from other clients start to block after a while.
>
> Rebooting the client named client resolves the issue.
There are some bugs around this functionality, but I *think* your
clients are new enough it shouldn't be an issue.
However, it's entirely possible your clients are actually making use
of enough inodes that the MDS server is running into its default
limits. If your MDS has memory available, you probably want to
increase the cache size from its default 100k (mds cache size = X).
Or maybe your kernels are too old; Zheng would know.
-Greg
>
> Clients are a mix of CentOS6 & CentOS7 running kernel
> 4.1.4-1.el7.elrepo.x86_64
> 4.1.4-1.el6.elrepo.x86_64
> 4.4.0-2.el6.elrepo.x86_64
> but other releases show the same behavior.
>
> Currently running 3 OSD Nodes and 3 combined MDS/MON-Nodes.
>
> What information do you need to further track down this issue? Quite
> unsure so this is only a rough overview of the setup.
>
>
> We have another issue with sometimes broken files ; bad checksums after
> storage, but I think I will start a new thread for this ;-)
>
> Thanks!
>
> --
> Kind Regards
> Michael
> _______________________________________________
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
