On 26 October 2015 at 14:54, Krutika Dhananjay <kdhan...@redhat.com> wrote:
> > Hi Lindsay, > > Thank you for trying out sharding and for your feedback. :) Please find my > comments inline. > Hi Krutika, thanks for the feed back. > With block size as low as 4MB, to the replicate module, these individual > shards appear as large number of small(er) files, effectively turning it > into some form of a small-file workload. > There is an enhancement being worked on in AFR by Pranith, which attempts > to improve write performance which will especially be useful when used with > sharding. That should make this problem go away. > Cool, also for my purposes (VM Image hosting), block sizes of 512MB are just as good and improve things considerably. > > One Bug: > After heals completed I shut down the VM's and run a MD5SUM on the VM > image (via glusterfs) on each nodes. They all matched except for one time > on gn3. Once I unmounted/remounted the datastore on gn3 the md5sum matched. > > > This could possibly be the effect of a caching bug reported at > https://bugzilla.redhat.com/show_bug.cgi?id=1272986. The fix is out for > review and I'm confident that it will make it into 3.7.6. > Cool, I can replicate it fairly reliable at the moment. Would it occur when using qemu/gfapi direct? > > > > One Oddity: > gluster volume heals datastore info *always* shows a split brain on the > directory, but it always heals without intervention. Dunno if this is > normal on not. > > > Which directory would this be? > Oddly it was the .shard directory > Do you have the glustershd logs? > Sorry no, and I haven't managed to replicate it again. Will keep trying. > Here is some documentation on sharding: > https://gluster.readthedocs.org/en/release-3.7.0/Features/shard/. Let me > know if you have more questions, and I will be happy to answer them. > The problems we foresaw with too many 4MB shards is that > i. entry self-heal under /.shard could result in complete crawl of the > /.shard directory during heal, or > ii. a disk replacement could involve lot many files needing to be created > and healed to the sink brick, > both of which would result in slower "entry" heal and rather high resource > consumption from self-heal daemon. > Thanks, most interesting reading. > Fortunately, with the introduction of more granular changelogs in > replicate module to identify exactly what files under a given directory > need to be healed to the sink brick, these problems should go away. > In fact this enhancement is being worked upon as we speak and is targeted > to be out by 3.8. Here is some doc: > http://review.gluster.org/#/c/12257/1/in_progress/afr-self-heal-improvements.md > (read section "Granular entry self-heals"). > That look very interesting - in fact from my point of view, it replaces the need for sharding altogether, that being the speed of heals. > > Yes. So Paul Cuzner and Satheesaran who have been testing sharding here > have reported better write performance with 512M shards. I'd be interested > to know what you feel about performance with relatively larger shards > (think 512M). > Seq Read speeds basically tripled, and seq writes improved to the limit of the network connection. Cheers, -- Lindsay
_______________________________________________ Gluster-users mailing list Gluster-users@gluster.org http://www.gluster.org/mailman/listinfo/gluster-users