FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-22 Thread Attilio Rao
[ Please CC me as I'm not subscribed to FreeBSD mailing lists ] Recently Bryan Drewery and I have been looking at this issue, in particular after that some people has been pointing us to DragonflyBSD / Linux benchmarks. Usually DB workloads are interesting mostly because they can expose some real

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Palle Girgensohn
21 maj 2014 kl. 22:05 skrev Sean Chittenden : I did some tests with zfs, and results where appallingly bad, but that was with db size > ram. > > I think the model used by PostgreSQL, as most databases, are very disk > block centric. Using zfs makes it hard to get good p

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Sean Chittenden
>>> I did some tests with zfs, and results where appallingly bad, but that was >>> with db size > ram. I think the model used by PostgreSQL, as most databases, are very disk block centric. Using zfs makes it hard to get good performance. But this was some time ago, maybe th

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Matthew Seaman
On 21/05/2014 18:17, Sean Chittenden wrote: >> I did some tests with zfs, and results where appallingly bad, but that was >> with db size > ram. >> > >> > I think the model used by PostgreSQL, as most databases, are very disk >> > block centric. Using zfs makes it hard to get good performance.

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Sean Chittenden
> I did some tests with zfs, and results where appallingly bad, but that was > with db size > ram. > > I think the model used by PostgreSQL, as most databases, are very disk block > centric. Using zfs makes it hard to get good performance. But this was some > time ago, maybe things have improv

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Palle Girgensohn
I did some tests with zfs, and results where appallingly bad, but that was with db size > ram. I think the model used by PostgreSQL, as most databases, are very disk block centric. Using zfs makes it hard to get good performance. But this was some time ago, maybe things have improved. Palle

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-21 Thread Gezeala M . Bacuño II
Gotcha. I've been testing using pgbench on FreeBSD 9.0 release + ZFS + pg 9.3.. I can reach the freebsd 10 stats on the pdf files if the dataset RAM. Test was done on pools without L2ARC and with/without compression. I also remember increasing a vm.pmap sysctl. I'm out of the office right now sick

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-20 Thread Palle Girgensohn
I got no response about how to grab performance data. The PostgreSQL team is also making an effort by setting up machines dedicated to performance measuring and tuning. And freebsd guys and PostgreSQL guys are apparently meeting at pgcon this week. We'll see where that leads. In the mean t

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-05-20 Thread Gezeala M . Bacuño II
Do you guys have any updates on this? -- regards gezeala bacuño II On Tue, Apr 22, 2014 at 11:52 PM, Palle Girgensohn wrote: > > > > 23 apr 2014 kl. 01:04 skrev Adrian Chadd : > > > > Hi, > > > > Are you able to repeat these tests (for both 9.2 and 9.3) whilst > > grabbing some performance da

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-04-22 Thread Palle Girgensohn
> 23 apr 2014 kl. 01:04 skrev Adrian Chadd : > > Hi, > > Are you able to repeat these tests (for both 9.2 and 9.3) whilst > grabbing some performance data from lock profiling and hwpmc? I sure can, but I'd love some pointers as to how this is done. Please? :-) > > The benchmarking is great b

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-04-22 Thread Adrian Chadd
Hi, Are you able to repeat these tests (for both 9.2 and 9.3) whilst grabbing some performance data from lock profiling and hwpmc? The benchmarking is great but it doesn't tell us enough information as to "why" things behave poorly compared to Linux and why the mmap drop isn't so great. What abo

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-04-21 Thread Palle Girgensohn
Den torsdagen den 20:e mars 2014 kl. 00:33:10 UTC+1 skrev Sean Chittenden: > > > As far as I know, the test was done on both UFS2 and ZFS and the > > difference was marginal. > > As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in > PostgreSQL 9.3, shared buffers are alloc

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-19 Thread Sean Chittenden
> As far as I know, the test was done on both UFS2 and ZFS and the > difference was marginal. As Adrian pointed out, there is an mmap(2) mutex in the way. Starting in PostgreSQL 9.3, shared buffers are allocated out of mmap(2) instead of shm. shm is only used to notify the PostgreSQL postmaster

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-19 Thread Petr Janda
Hi, As far as I know, the test was done on both UFS2 and ZFS and the difference was marginal. Petr On 18/03/2014 10:47 PM, Steven Hartland wrote: > - Original Message - From: "Petr Janda" > > >> Hi guys, >> >> Just want to share these pgbench results done by DragonFlyBSD, and would >>

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-18 Thread Adrian Chadd
Hi, the pgsql testing done has been analysed by a few developers. The TL;DR version is that there's significant lock contention in the VM / mmap path and it sticks out like a sore thumb when one does lock profiling. -a On 18 March 2014 05:00, Matthew Seaman wrote: > On 03/18/14 03:12, Petr Ja

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-18 Thread Matthew Seaman
On 03/18/14 03:12, Petr Janda wrote: > ust want to share these pgbench results done by DragonFlyBSD, and would > like some input on why these numbers look so bad and what can be done to > improve (ie. kernel tunables etc) the performance. > > http://lists.dragonflybsd.org/pipermail/users/attachmen

Re: FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-18 Thread Steven Hartland
- Original Message - From: "Petr Janda" Hi guys, Just want to share these pgbench results done by DragonFlyBSD, and would like some input on why these numbers look so bad and what can be done to improve (ie. kernel tunables etc) the performance. http://lists.dragonflybsd.org/piperma

FreeBSD 10 and PostgreSQL 9.3 scalability issues

2014-03-18 Thread Petr Janda
Hi guys, Just want to share these pgbench results done by DragonFlyBSD, and would like some input on why these numbers look so bad and what can be done to improve (ie. kernel tunables etc) the performance. http://lists.dragonflybsd.org/pipermail/users/attachments/20140310/4250b961/attachment-0001