[zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Iwan Aucamp
I'm getting sub-optimal performance with an mmap based database (mongodb) which is running on zfs of Solaris 10u9. System is Sun-Fire X4270-M2 with 2xX5680 and 72GB (6 * 8GB + 6 * 4GB) ram (installed so it runs at 1333MHz) and 2 * 300GB 15K RPM disks - a few mongodb instances are running wit

[zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Iwan Aucamp
On 05/28/2012 10:12 PM, Andrew Gabriel wrote: On 05/28/12 20:06, Iwan Aucamp wrote: I'm thinking of doing the following: - relocating mmaped (mongo) data to a zfs filesystem with only metadata cache - reducing zfs arc cache to 16 GB Is there any other recommendations - and is above likely

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Andrew Gabriel
On 05/28/12 20:06, Iwan Aucamp wrote: I'm getting sub-optimal performance with an mmap based database (mongodb) which is running on zfs of Solaris 10u9. System is Sun-Fire X4270-M2 with 2xX5680 and 72GB (6 * 8GB + 6 * 4GB) ram (installed so it runs at 1333MHz) and 2 * 300GB 15K RPM disks -

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Lionel Cons
On Mon, May 28, 2012 at 9:06 PM, Iwan Aucamp wrote: > I'm getting sub-optimal performance with an mmap based database (mongodb) > which is running on zfs of Solaris 10u9. > > System is Sun-Fire X4270-M2 with 2xX5680 and 72GB (6 * 8GB + 6 * 4GB) ram > (installed so it runs at 1333MHz) and 2 * 300GB

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Richard Elling
On May 28, 2012, at 12:46 PM, Lionel Cons wrote: > On Mon, May 28, 2012 at 9:06 PM, Iwan Aucamp wrote: >> I'm getting sub-optimal performance with an mmap based database (mongodb) >> which is running on zfs of Solaris 10u9. >> >> System is Sun-Fire X4270-M2 with 2xX5680 and 72GB (6 * 8GB + 6 * 4

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Richard Elling
question below... On May 28, 2012, at 1:25 PM, Iwan Aucamp wrote: > On 05/28/2012 10:12 PM, Andrew Gabriel wrote: >> On 05/28/12 20:06, Iwan Aucamp wrote: >>> I'm thinking of doing the following: >>> - relocating mmaped (mongo) data to a zfs filesystem with only >>> metadata cache >>> - reduci

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Lionel Cons
On 28 May 2012 22:10, Richard Elling wrote: > The only recommendation which will lead to results is to use a > different OS or filesystem. Your choices are > - FreeBSD with ZFS > - Linux with BTRFS > - Solaris with QFS > - Solaris with UFS > - Solaris with NFSv4, use ZFS on independent fileserver

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Richard Elling
[Apologies to the list, this has expanded past ZFS, if someone complains, we can move the thread to another illumos dev list] On May 28, 2012, at 2:18 PM, Lionel Cons wrote: > On 28 May 2012 22:10, Richard Elling wrote: >> The only recommendation which will lead to results is to use a >> differe

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Jim Klimov
2012-05-29 0:34, Richard Elling wrote: I'd be interested in the results of such tests. You can change the primarycache parameter on the fly, so you could test it in less time than it takes for me to type this email :-) I believe it would also take some time for memory distribution to settle, e

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-28 Thread Daniel Carosone
On Mon, May 28, 2012 at 01:34:18PM -0700, Richard Elling wrote: > I'd be interested in the results of such tests. Me too, especially for databases like postgresql where there's a complementary cache size tunable within the db that often needs to be turned up, since they implicitly rely on some fi

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-29 Thread Iwan Aucamp
On 05/29/2012 03:29 AM, Daniel Carosone wrote: For the mmap case: does the ARC keep a separate copy, or does the vm system map the same page into the process's address space? If a separate copy is made, that seems like a potential source of many kinds of problems - if it's the same page then th

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-05-30 Thread Bob Friesenhahn
On Tue, 29 May 2012, Iwan Aucamp wrote:  - Is there a  parameter similar to /proc/sys/vm/swappiness that can control how long unused pages in page cache stay in physical ram if there is no shortage of physical ram ? And if not how long will unused pages stay in page cache stay in physical ram g

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-06-01 Thread Jeff Bacon
> I'm getting sub-optimal performance with an mmap based database > (mongodb) which is running on zfs of Solaris 10u9. > > System is Sun-Fire X4270-M2 with 2xX5680 and 72GB (6 * 8GB + 6 * > 4GB) > ram (installed so it runs at 1333MHz) and 2 * 300GB 15K RPM disks > > - a few mongodb instances ar

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-06-01 Thread Jeff Bacon
> I'd be interested in the results of such tests. You can change the > primarycache > parameter on the fly, so you could test it in less time than it > takes for me to type this email :-) > -- Richard Tried that. Performance headed south like a cat with its tail on fire. We didn't bother quanti

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-06-01 Thread Jeff Bacon
> Anybody who has worked on a SPARC system for the past 15 years is well > aware of NUMAness. We've been living in a NUMA world for a very long time, > a world where the processors were slow and far memory latency is much, much > worse than we see in the x86 world. > > I look forward to seeing the

Re: [zfs-discuss] Remedies for suboptimal mmap performance on zfs

2012-06-01 Thread Iwan Aucamp
On 06/01/2012 02:33 PM, Jeff Bacon wrote: I'd be interested in the results of such tests. You can change the primarycache parameter on the fly, so you could test it in less time than it takes for me to type this email :-) -- Richard Tried that. Performance headed south like a cat with its tail