Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
On Sat, 19 Apr 2008, michael schuster wrote: > that's true most of the time ... unless free memory gets *really* low, then > Solaris *does* start to swap (ie page out pages by process). IIRC, the > threshold for swapping is minfree (measured in pages), and the value that > needs to fall below this threshold is freemem. Most people here are likely too young to know what "swapping" really is. Swapping is not the same as the paging that Solaris does. With swapping the kernel knows that this address region belongs to this process and we are short of RAM so block copy the process to the swap area, and only remember that it exists via the process table. Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Bob Friesenhahn wrote: > On Sun, 20 Apr 2008, A Darren Dunham wrote: >> I think these paragraphs are referring to two different concepts with >> "swap". Swapfiles or backing store in the first, and virtual memory >> space in the second. > > The "swap" area is mis-named since Solaris never "swaps". Some older > operating systems would put an entire program in the swap area when > the system ran short on memory and would have to "swap" between > programs. Solaris just "pages" (a virtual memory function) and it is > very smart about how and when it does it. Only dirty pages which are > not write-mapped to a file in the filesystem need to go in the swap > area, and only when the system runs short on RAM. that's true most of the time ... unless free memory gets *really* low, then Solaris *does* start to swap (ie page out pages by process). IIRC, the threshold for swapping is minfree (measured in pages), and the value that needs to fall below this threshold is freemem. HTH Michael -- Michael Schuster http://blogs.sun.com/recursion Recursion, n.: see 'Recursion' ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
On Sun, 20 Apr 2008, A Darren Dunham wrote: > > I think these paragraphs are referring to two different concepts with > "swap". Swapfiles or backing store in the first, and virtual memory > space in the second. The "swap" area is mis-named since Solaris never "swaps". Some older operating systems would put an entire program in the swap area when the system ran short on memory and would have to "swap" between programs. Solaris just "pages" (a virtual memory function) and it is very smart about how and when it does it. Only dirty pages which are not write-mapped to a file in the filesystem need to go in the swap area, and only when the system runs short on RAM. Solaris is a quite-intensely memory-mapped system. The memory mapping allows a huge amount of sharing of shared library files, program text images, and unmodified pages shared after fork(). The end result is a very memory-efficient OS. Now if we could just get ZFS ARC and Gnome Desktop to not use any memory, we would be in nirvana. :-) Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Solaris 10U5 ZFS features?
I would hope at least it has that giant FSYNC patch for ZFS already present? We ran into this issue and it nearly killed Solaris here in our Data Center as a product it was such a bad experience. Fix was in 127728 (x86) and 127729 (Sparc). Well anyhow good to see U5 is out, hadn't even heard about it so downloading now. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
On Sat, Apr 19, 2008 at 10:28:45AM -0700, Richard Elling wrote: > Bob Friesenhahn wrote: > > I don't agree that if swap is used that performance will necessarily > > suck. If swap is available, Solaris will mount /tmp there, which > > helps temporary file performance. It is best to look at system paging > > (hard faults) while programs are running in order to determine if > > performance sucks due to inadequate RAM. In many runtime > > environments, only a small bit of the application address space is > > ever needed. > > swapfs is always there. But, IMHO, it is a misnomer because it just uses > the virtual memory system. Why a misnomer? "swap" and "virtual memory" are used as identical terms in many places in Solaris. But since /tmp was mentioned, perhaps you're referring to tmpfs instead of swapfs? -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
On Sat, Apr 19, 2008 at 12:16:11PM -0500, Bob Friesenhahn wrote: > On Sat, 19 Apr 2008, Richard Elling wrote: > > > > Don't worry about swapping on CF. In most cases, you won't be > > using the swap device for normal operations. You can use the > > swap -l command to observe the swap device usage. No usage > > means that you can probably do away with it. If you actually > > use swap, performance will suck, so buy more RAM. > > I don't agree that if swap is used that performance will necessarily > suck. If swap is available, Solaris will mount /tmp there, which > helps temporary file performance. I think these paragraphs are referring to two different concepts with "swap". Swapfiles or backing store in the first, and virtual memory space in the second. -- Darren Dunham [EMAIL PROTECTED] Senior Technical Consultant TAOShttp://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Bob Friesenhahn wrote: > On Sat, 19 Apr 2008, Richard Elling wrote: > >> Don't worry about swapping on CF. In most cases, you won't be >> using the swap device for normal operations. You can use the >> swap -l command to observe the swap device usage. No usage >> means that you can probably do away with it. If you actually >> use swap, performance will suck, so buy more RAM. >> > > I don't agree that if swap is used that performance will necessarily > suck. If swap is available, Solaris will mount /tmp there, which > helps temporary file performance. It is best to look at system paging > (hard faults) while programs are running in order to determine if > performance sucks due to inadequate RAM. In many runtime > environments, only a small bit of the application address space is > ever needed. > swapfs is always there. But, IMHO, it is a misnomer because it just uses the virtual memory system. The prevailing method of determining memory shortfall is to observe the page scanner (scan rate, sr). But just for grins, try swap -l on your systems and see if any pages have been used on the swap device. The answer usually surprises ;-) > More RAM definitely improves ZFS repeated read performance due to > caching in RAM. ZFS gives otherwise unused memory something useful to > do. > > Regardless, with a 64-bit CPU and a 64-bit OS it seems like a crying > shame to install less than 4GB of RAM. :-) > Yep, or if you do OpenGL stuff, like I've been doing lately, much more RAM :-) -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] how to measure in kernel how much time one operation takes
shri wrote: > what call to be used in kernel to measure time taken for a operation if time > quantum i s smaller than tick(frequency). > i tried using ddi_get_lbolt at time t1 and t2.(at time t1 operation started > and t2 it finished) but it always gives me zero(t2-t1). > > Try get hi-res time, gethrtime() http://docs.sun.com/app/docs/doc/816-5180/gethrtime-9f?a=view -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
On Sat, 19 Apr 2008, Richard Elling wrote: > > Don't worry about swapping on CF. In most cases, you won't be > using the swap device for normal operations. You can use the > swap -l command to observe the swap device usage. No usage > means that you can probably do away with it. If you actually > use swap, performance will suck, so buy more RAM. I don't agree that if swap is used that performance will necessarily suck. If swap is available, Solaris will mount /tmp there, which helps temporary file performance. It is best to look at system paging (hard faults) while programs are running in order to determine if performance sucks due to inadequate RAM. In many runtime environments, only a small bit of the application address space is ever needed. More RAM definitely improves ZFS repeated read performance due to caching in RAM. ZFS gives otherwise unused memory something useful to do. Regardless, with a 64-bit CPU and a 64-bit OS it seems like a crying shame to install less than 4GB of RAM. :-) Bob == Bob Friesenhahn [EMAIL PROTECTED], http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Pascal Vandeputte wrote: > I see. I'll only be running a minimal Solaris install with ZFS and samba on > this machine, so I wouldn't expect immediate memory issues with 2 gigabytes > of RAM. OTOH I read that ZFS is a real memory hog so I'll be careful. > Memory usage is completely dependent on the workload. Unless you are doing a *lot* of writes with a slow back end (hmmm) then you should be ok with modest RAM. > I've tested swap on a ZFS volume now, it's really easy so I'll try running > without swap for some quick performance testing and use swap on ZFS after > that. This also takes away my fears about using a swap slice on the > CompactFlash card I'll be booting from. > > To save you some grief, please wait for b88 before swapping to ZFS. Don't worry about swapping on CF. In most cases, you won't be using the swap device for normal operations. You can use the swap -l command to observe the swap device usage. No usage means that you can probably do away with it. If you actually use swap, performance will suck, so buy more RAM. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Pascal Vandeputte wrote: > Thanks a lot for your input, I understand those numbers a lot better now! > I'll look deeper into hardware issues. It's a pity that I can't get older > BIOS versions flashed. But I've got some other hardware lying around. > > Someone suggested lowering the 35 iops default, but I can't find any > information anywhere on how to accomplish this (not with Google, not in the > ZFS Admin guide either). > It is in the Evil Tuning Guide. But don't bother, it won't fix your problem. The evidence suggests you get 10ms response even with only 1 iop queued to the device. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Thanks a lot for your input, I understand those numbers a lot better now! I'll look deeper into hardware issues. It's a pity that I can't get older BIOS versions flashed. But I've got some other hardware lying around. Someone suggested lowering the 35 iops default, but I can't find any information anywhere on how to accomplish this (not with Google, not in the ZFS Admin guide either). Greetings, Pascal This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Great, superb write speeds with a similar setup, my motivation is growing again ;-) It just occurs to me that I have a spare Silicon Image 3124 SATA card lying around. I was postponing testing of these drives on my desktop because it has an Intel ICH9 SATA controller probably quite similar to the ICH9R (RAID support) in my Solaris box, but that 3124 may give completely different results with the Seagates. Test coming up. (the forum seems to be having technical difficulties, I hope my replies end up in the right places...) This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
(the lt and gt symbols are filtered by the forum I guess; replaced with minus signs now) # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 -DEFAULT cyl 45597 alt 2 hd 255 sec 126- /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 1. c1t1d0 -ATA-ST3750330AS-SD15-698.64GB- /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 2. c1t2d0 -ATA-ST3750330AS-SD15-698.64GB- /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Thanks, I'll try installing Solaris on a 1GB CF card in an CF-to-IDE adapter, so all disks will then be completely available to ZFS. Then I needn't worry about different size block devices either. I also find it weird that the boot disk is displayed differently from the other two disks if I run the "format" command... (could be normal though, as I said before I'm new to Solaris) # format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c1t0d0 /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 1. c1t1d0 /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 2. c1t2d0 /[EMAIL PROTECTED],0/pci8086,[EMAIL PROTECTED],2/[EMAIL PROTECTED],0 This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
I see. I'll only be running a minimal Solaris install with ZFS and samba on this machine, so I wouldn't expect immediate memory issues with 2 gigabytes of RAM. OTOH I read that ZFS is a real memory hog so I'll be careful. I've tested swap on a ZFS volume now, it's really easy so I'll try running without swap for some quick performance testing and use swap on ZFS after that. This also takes away my fears about using a swap slice on the CompactFlash card I'll be booting from. Thanks! This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS raidz write performance:what to expect from SATA drives on ICH9R
Pascal Vandeputte hotmail.com> writes: > > I'm at a loss, I'm thinking about just settling for the 20MB/s write > speeds with a 3-drive raidz and enjoy life... As Richard Elling pointed out, the ~10ms per IO operation implies seeking, or hardware/firmware problems. The mere fact you observed a low 27 MB/s sequential write throughput on c1t0d0s0 indicates this is not a ZFS pb. Test other disks, another SATA controller, mobo, BIOS/firmware, etc. As you pointed out, these disks should normally be capable of a 80-90 MB/s write throughput. Like you I would also expect ~100 MB/s writes on a 3-drive raidz pool. As a datapoint, I see 150 MB/s writes on a 4-drive raidz on a similar config (750GB SATA Samsung HD753LJ disks, SB600 AHCI controller, low-end CPU). -marc ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
