On Fri, Apr 28, 2017 at 3:44 PM, Szymon Miotk <szymon.mi...@gmail.com> wrote:
> Dear Gluster community, > > I have problems with tuning Gluster for small files performance on SSD. > > My usage scenario is, as I've learned, worst possible scenario, but > it's not up to me to change it: > - small 1KB files > - at least 20M of those > - approx. 10 files/directory > - mostly writes > - average speed 1000 files/sec with peaks up to 10K files/sec. > > I'm doing something wrong, because I cannot get past performance metrics > 4K files/sec for distributed volume (2 bricks) > 2K files/sec for replicated volume (2 bricks). > I've been experimenting with various XFS formatting and mounting > options and with Gluster tuning (md-cache, lookup optimize, thread, > writeback, tiering), but no luck. > > I've learned that it's not disk IO that is the bottleneck (direct > tests on mounted XFS partition show waaay better results, like 100K > files/sec). > > As I've learned from > http://blog.gluster.org/2014/03/experiments-using-ssds-with-gluster/ > it's possible to get 24K files/sec performance (and that was three years > ago). > > How many clients are you running? Considering Gluster is a distributed solution, the performance should be measured as aggregate of all the clients. > Test I'm using, run on one server (2 x Xeons, 256 GB RAM, 10GbE network): > smallfile_cli.py --operation create --threads 32 --file-size 1 --files > 15625 --top /mnt/testdir/test > > Setup: > 2 servers with 2 Xeons each, 256GB RAM, 8 x 800GB SSD drives in RAID6, > 10GbE network > Ubuntu 14.04 > Gluster 3.7.3 > > Do you have any hints where I should start investigating for the > bottleneck? > > Currently just the fuse mount may be the bottleneck, but I recommend running multiple clients (from different machines) doing these operations in parallel to get the best results out of Gluster. -Amar Best regards, > Szymon > _______________________________________________ > Gluster-users mailing list > Gluster-users@gluster.org > http://lists.gluster.org/mailman/listinfo/gluster-users > -- Amar Tumballi (amarts)
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://lists.gluster.org/mailman/listinfo/gluster-users