Hi Jörn, On Fri, Mar 29, 2019 at 5:20 AM Clausen, Jörn <jclau...@geomar.de> wrote: > > Hi! > > In my ongoing quest to wrap my head around Ceph, I created a CephFS > (data and metadata pool with replicated size 3, 128 pgs each).
What version? > When I > mount it on my test client, I see a usable space of ~500 GB, which I > guess is okay for the raw capacity of 1.6 TiB I have in my OSDs. > > I run bonnie with > > -s 0G -n 20480:1k:1:8192 > > i.e. I should end up with ~20 million files, each file 1k in size > maximum. After about 8 million files (about 4.7 GBytes of actual use), > my cluster runs out of space. Meaning, you got ENOSPC? > Is there something like a "block size" in CephFS? I've read > > http://docs.ceph.com/docs/master/cephfs/file-layouts/ > > and thought maybe object_size is something I can tune, but I only get > > $ setfattr -n ceph.dir.layout.object_size -v 524288 bonnie > setfattr: bonnie: Invalid argument You can only set a layout on an empty directory. The layouts here are not likely to be the cause. > Is this even the right approach? Or are "CephFS" and "many small files" > such opposing concepts that it is simply not worth the effort? You should not have had issues growing to that number of files. Please post more information about your cluster including configuration changes and `ceph osd df`. -- Patrick Donnelly _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
