Re: [Gluster-users] Segfaults after upgrade to GlusterFS 3.10.9

2018-01-19 Thread Niklas Hambüchen
What's /proc/sys/kernel/core_pattern set to for you? For me it is % cat /proc/sys/kernel/core_pattern core which will drop a core file in the working directory of the process. On 19/01/2018 12.56, Frank Wall wrote: > Is there something I could do to trigger the creation of core files?

Re: [Gluster-users] create volume in two different Data Centers

2017-10-24 Thread Niklas Hambüchen
On 24/10/17 13:01, Alessandro Briosi wrote: > I would set up a VPN (tinc could work well). I, too, would recommend to try tinc for this, it can automatically route traffic of nodes that don't have direct access to other nodes via those nodes that do. I have a publicly available setup of Gluster

Re: [Gluster-users] Confusing lstat() performance

2017-09-20 Thread Niklas Hambüchen
Note, I have now put out a patch for bup that makes it 50x faster on my 1M files on gluster, exploiting the fact that stat() is fast after getdents() on it: https://groups.google.com/forum/#!topic/bup-list/6CM7Jr7b_Ek I'd still be interested in answers to this question though: > What caching

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
On 18/09/17 17:23, Ben Turner wrote: > Do you want tuned or untuned? If tuned I'd like to try one of my tunings for > metadata, but I will use yours if you want. (Re-CC'd list) I would be interested in both, if possible: To confirm that it's not only my machines that exhibit this behaviour

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
On 18/09/17 16:51, Ben Turner wrote: > I wouldn't mind, but I don't have your dataset. Oh sorry, I thought I had posted that here but in fact I did so in a different issue regarding getdents() performance (bug 1478411). My benchmarking data set is trivial: 100k empty files. In a directory on

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
Hi Ben, do you know if the smallfile benchmark also does interleaved getdents() and lstat, which is what I found as being the key difference that creates the performance gap (further down this thread)? Also, wouldn't `--threads 8` change the performance numbers by factor 8 versus the plain `ls`

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
I found the reason now, at least for this set of lstat()s I was looking at. bup first does all getdents(), obtaining all file names in the directory, and then stat()s them. Apparently this destroys some of gluster's caching, making stat()s ~100x slower. What caching could this be, and how could

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
On 17/09/17 18:03, Niklas Hambüchen wrote: > So far the only difference between `ls` and `bup index` I could observe > is that `bup index` chdir()s into the directory to index, ls doesn't. > > But when I `cd` into the dir and run `ls` without directory argument, it > is still much

Re: [Gluster-users] Confusing lstat() performance

2017-09-19 Thread Niklas Hambüchen
On 15/09/17 03:46, Niklas Hambüchen wrote: >> Out of interest have you tried testing performance >> with performance.stat-prefetch enabled? I have now tested with `performance.stat-prefetch: on` but am not observing a difference. So far the only difference between `ls` and `bup in

Re: [Gluster-users] Confusing lstat() performance

2017-09-18 Thread Niklas Hambüchen
On 15/09/17 02:45, Sam McLeod wrote: > Out of interest have you tried testing performance > with performance.stat-prefetch enabled? Not yet, because I'm still struggling to understand the current more basic setup's performance behaviour (with it being off), but it's definitely on my list and I'll

[Gluster-users] Confusing lstat() performance

2017-09-14 Thread Niklas Hambüchen
Hi, I have a gluster 3.10 volume with a dir with ~1 million small files in them, say mounted at /mnt/dir with FUSE, and I'm observing something weird: When I list and stat them all using rsync, then the lstat() calls that rsync does are incredibly fast (23 microseconds per call on average,

Re: [Gluster-users] Issue in installing Gluster 3.9.0

2017-02-24 Thread Niklas Hambüchen
I also found that the Ubuntu PPAs maintained by the gluster team, when unpacked, contain a patch in the debian/patches directory that addresses these issues (but of course it'd be better to have it fixed upstream). On 22/02/17 18:42, Shyam wrote: > Optionally try patching the sources with this