Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD
On Mon, Apr 21, 2014 at 05:43:46PM +0200, Andres Freund wrote: > > On 2014-04-21 17:39:39 +0200, Magnus Hagander wrote: > > But do we really want a *guc* for it though? Isn't it enough (and in fact > > better) with a configure switch to pick the implementation when multiple > > are available, that could then be set by default for example by the freebsd > > ports build? That's a lot less "overhead" to keep dragging around... > > Well, we don't know at all it's just freebsd that's affected. In fact, I > would be surprised if there aren't other platforms that regressed due to > this. The only platforms affected are the BSDs. I ran (or tried to run) Pgbench on the four major ones during the last two years. My experience so far: - FreeBSD: has trouble scaling; there's some interest to improve things but nobody has done anything about it yet - NetBSD: crashes under load; this could have been fixed but when I ran the benchmarks in 2012 none of the developers seemed to care. - OpenBSD: couldn't run the benchmark; for some reason this operating system is unable to mmap() 32GB of memory on a recent PC with 128GB of RAM. - DragonFly: scales better than everything else out there even with mmap() Given these results I doubt reintroducing SysV shm memory use on PostgreSQL is worthwile; most platforms where it has a performance impact have much bigger issues to fix first. -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD
On Sun, Apr 20, 2014 at 04:07:25PM +0200, Palle Girgensohn wrote: > > >> If mmap needs to perform well in the kernel, I'd like to know of someone > >> with FreeBSD kernel knowledge who is interested in working with mmap > >> perfocmance. If mmap is indeed the cuplrit, I've just tested 9.2.8 vs > >> 9.3.4, I nevere isolated the mmap patch, although I believe Francois did > >> just that with similar results. > > > > I did test the 9.3 -devel branch before and after the SysV shared memory => > > mmap commit. The performance degradation was visible. > > > > I recently ran a few benchmarks of PostgreSQL 9.3 with different operating > > systems > > including DragonFly 3.6 and FreeBSD 10. You may be interested in the > > results: > > > > http://lists.dragonflybsd.org/pipermail/users/2014-March/128216.html > > > > Interesting, indeed. The fixes to dragonfly where made quite recently, in > 3.2, right? The most important fixes occured in the 3.1 development version, around September 2012. There was definitely more than an isolated patch; the new scheduler was only part of the performance improvements. I'm afraid none of the commits would be applicable as-is to FreeBSD 10.x; the DragonFly kernel is vastly different in locking, threading and VM management. The FreeBSD folks should know what to do though; I collected performance counter data during the last benchmark run and sent it to adrian@. It was also discussed on freebsd-performance; the thread begins here: http://lists.freebsd.org/pipermail/freebsd-performance/2014-March/004770.html -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Perfomance degradation 9.3 (vs 9.2) for FreeBSD
Hi, On Sun, Apr 20, 2014 at 11:24:38AM +0200, Palle Girgensohn wrote: > > I see performance degradation with PostgreSQL 9.3 vs 9.2 on FreeBSD, and I'm > wondering who to poke to mitigate the problem. In reference to this thread > [1], who where the FreeBSD people that Francois mentioned? At least one FreeBSD hacker came to discuss it on the #dragonflybsd irc channel and tried to run the benchmark on a 80-core machine. I didn't keep logs and don't remember his/their name(s) but there was definitely some FreeBSD effort at the time to investigate and fix things. > If mmap needs to perform well in the kernel, I'd like to know of someone with > FreeBSD kernel knowledge who is interested in working with mmap perfocmance. > If mmap is indeed the cuplrit, I've just tested 9.2.8 vs 9.3.4, I nevere > isolated the mmap patch, although I believe Francois did just that with > similar results. I did test the 9.3 -devel branch before and after the SysV shared memory => mmap commit. The performance degradation was visible. I recently ran a few benchmarks of PostgreSQL 9.3 with different operating systems including DragonFly 3.6 and FreeBSD 10. You may be interested in the results: http://lists.dragonflybsd.org/pipermail/users/2014-March/128216.html -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SYSV shared memory vs mmap performance
Hi, On Mon, Jan 28, 2013 at 03:27:28PM +, YAMAMOTO Takashi wrote: > > can you give me a pointer? This bug report for a start: http://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=46833 This is the only one I've filled; I also remember having irc discussions with some netbsd developers about the system getting unresponsive under load. -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SYSV shared memory vs mmap performance
On Fri, Jan 25, 2013 at 05:55:19PM -0500, Robert Haas wrote: > On Fri, Jan 25, 2013 at 9:38 AM, Francois Tigeot wrote: > > > > Some links with more details about improvements and final results: > > http://www.shiningsilence.com/dbsdlog/2012/09/19/10403.html > > http://www.shiningsilence.com/dbsdlog/2012/10/11/10544.html > > http://www.dragonflybsd.org/performance/ > > Well, that looks pretty cool. Is there anything we can sensibly do to > recover the lost performance on FreeBSD and NetBSD? Apart from reintroducing SYSV shm usage, I don't think the Postgres team can do much about it. Mmap really needs kernel work to perform well. Some FreeBSD people are looking into the issue and have started to run benchmarks of their own to find out what needs to be fixed; I guess they will adapt in time. I'm less optimistic on the NetBSD front: even though I reported major show-stopper bugs (system died under load and was unable to complete a pgbench run), no one seemed to care. -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] SYSV shared memory vs mmap performance
Hi, Robert Haas wrote: On Fri, Jan 25, 2013 at 8:40 AM, Bruce Momjian wrote: Just a reminder we might have *BSD performance issues with our use of Posix shared memory in Postgres 9.3. I am attaching the PDF the user posted. This is a good point. The question which I believe I asked before and haven't gotten an answer to is whether there's some way to get the benefit of shm_use_phys with an anonymous mapping. There is. Postgres 9.3+mmap performance on DragonFly is now much better than these old benchmark results show. After the initial disappointing result, I went on a benchmarking/tuning binge with our Dear Leader Matt Dillon. Taking advantage of some previous cpu topology work from Mihai Carabas, he heavily improved most performance shortcomings we found in the DragonFly kernel. There were a few mail about this changes on the DragonFly mailing-lists and Justin Sherill wrote some interesting articles on his blog. Some links with more details about improvements and final results: http://www.shiningsilence.com/dbsdlog/2012/09/19/10403.html http://www.shiningsilence.com/dbsdlog/2012/10/11/10544.html http://www.dragonflybsd.org/performance/ It seems to me to be slightly insane to impose draconian shared memory limits out of the box and then complain when people switch to some other type of shared memory to get around them. I realize that Dragonfly may not be doing that (because I think they may have raised the default shared-memory limits), but I believe some of the more mainstream BSDs are. The original SYSV limits looked like something straight from the 1980s; we're now autotuning them on DragonFly. FreeBSD and NetBSD still needed manual tuning last time I had a look. -- Francois Tigeot -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
