On Mon, Mar 12, 2018 at 6:23 PM, Ondrej Valousek <
ondrej.valou...@s3group.com> wrote:
> Hi,
>
> Gluster will never perform well for small files.
>
> I believe there is nothing you can do with this.
>
It is bad compared to a disk filesystem but I believe it is much closer to
NFS now.
Andreas,
Looking at your workload, I am suspecting there to be lot of LOOKUPs
which reduce performance. Is it possible to do the following?
# gluster volume profile <volname> info incremental
#execute your workload
# gluster volume profile <volname> info incremental >
/path/to/file/that/you/need/to/send/us
If the last line in there is LOOKUP, mostly we need to enable nl-cache
feature and see how it performs.
> Ondrej
>
>
>
> *From:* gluster-users-boun...@gluster.org [mailto:gluster-users-bounces@
> gluster.org] *On Behalf Of *Andreas Ericsson
> *Sent:* Monday, March 12, 2018 1:47 PM
> *To:* Gluster-users@gluster.org
> *Subject:* [Gluster-users] Expected performance for WORM scenario
>
>
>
> Heya fellas.
>
>
>
> I've been struggling quite a lot to get glusterfs to perform even
> halfdecently with a write-intensive workload. Testnumbers are from gluster
> 3.10.7.
>
>
>
> We store a bunch of small files in a doubly-tiered sha1 hash fanout
> directory structure. The directories themselves aren't overly full. Most of
> the data we write to gluster is "write once, read probably never", so 99%
> of all operations are of the write variety.
>
>
>
> The network between servers is sound. 10gb network cards run over a 10gb
> (doh) switch. iperf reports 9.86Gbit/sec. ping reports a latency of 0.1 -
> 0.2 ms. There is no firewall, no packet inspection and no nothing between
> the servers, and the 10gb switch is the only path between the two machines,
> so traffic isn't going over some 2mbit wifi by accident.
>
>
>
> Our main storage has always been really slow (write speed of roughly
> 1.5MiB/s), but I had long attributed that to the extremely slow disks we
> use to back it, so now that we're expanding I set up a new gluster cluster
> with state of the art NVMe SSD drives to boost performance. However,
> performance only hopped up to around 2.1MiB/s. Perplexed, I tried it first
> with a 3-node cluster using 2GB ramdrives, which got me up to 2.4MiB/s. My
> last resort was to use a single node running on ramdisk, just to 100%
> exclude any network shenanigans, but the write performance stayed at an
> absolutely abysmal 3MiB/s.
>
>
>
> Writing straight to (the same) ramdisk gives me "normal" ramdisk speed (I
> don't actually remember the numbers, but my test that took 2 minutes with
> gluster completed before I had time to blink). Writing straight to the
> backing SSD drives gives me a throughput of 96MiB/sec.
>
>
>
> The test itself writes 8494 files that I simply took randomly from our
> production environment, comprising a total of 63.4MiB (so average file size
> is just under 8k. Most are actually close to 4k though, with the occasional
> 2-or-so MB file in there.
>
>
>
> I have googled and read a *lot* of performance-tuning guides, but the
> 3MiB/sec on single-node ramdisk seems to be far beyond the crippling one
> can cause by misconfiguration of a single system.
>
>
>
> With this in mind; What sort of write performance can one reasonably hope
> to get with gluster? Assume a 3-node cluster running on top of (small)
> ramdisks on a fast and stable network. Is it just a bad fit for our
> workload?
>
>
>
> /Andreas
>
> -----
>
> The information contained in this e-mail and in any attachments is
> confidential and is designated solely for the attention of the intended
> recipient(s). If you are not an intended recipient, you must not use,
> disclose, copy, distribute or retain this e-mail or any part thereof. If you
> have received this e-mail in error, please notify the sender by return e-mail
> and delete all copies of this e-mail from your computer system(s). Please
> direct any additional queries to: communicati...@s3group.com. Thank You.
> Silicon and Software Systems Limited (S3 Group). Registered in Ireland no.
> 378073. Registered Office: South County Business Park, Leopardstown, Dublin
> 18.
>
>
> _______________________________________________
> Gluster-users mailing list
> Gluster-users@gluster.org
> http://lists.gluster.org/mailman/listinfo/gluster-users
>
--
Pranith
_______________________________________________
Gluster-users mailing list
Gluster-users@gluster.org
http://lists.gluster.org/mailman/listinfo/gluster-users
