[zfs-discuss] Tunable parameter to zfs memory use
Hi been using now zfs since 06/06 u2 release has been out. one thing have notised. zfs eats a lot of memory. right after boot mem usage is about 280M but after accessing zfs disks usage rises fast to be 900M. and it seems to stay in level of 90% of tot mem. also noted it frees used mem but running heavy apps you see performace impact while zfs frees memory to other use. now idea: could there be /etc/system tunable parameter for zfs so we could manually set % of total memory zfs can use. like : zfs_max_phys_mem 30% would tell zfs that it can use 30% of available memory. and via versa zfs_min_phys_mem 10% would tell zfs that it can allways use 10% of available memory. -- these maybe need to global values for all zfs filesystems or just simple zfs option that can be set as other zfs options , like compression, etc... then this setting could be easily per zfs filesystem Thanks This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Tunable parameter to zfs memory use
homerun wrote: been using now zfs since 06/06 u2 release has been out. one thing have notised. zfs eats a lot of memory. right after boot mem usage is about 280M but after accessing zfs disks usage rises fast to be 900M. and it seems to stay in level of 90% of tot mem. also noted it frees used mem but running heavy apps you see performace impact while zfs frees memory to other use. Yes, that is the case. As I understand things it's an artefact of how zfs is implemented. There are changes in the Solaris 10 patch pipeline which should make the kmem usage somewhat better. now idea: could there be /etc/system tunable parameter for zfs so we could manually set % of total memory zfs can use. like : zfs_max_phys_mem 30% would tell zfs that it can use 30% of available memory. and via versa zfs_min_phys_mem 10% would tell zfs that it can allways use 10% of available memory. -- these maybe need to global values for all zfs filesystems or just simple zfs option that can be set as other zfs options , like compression, etc... then this setting could be easily per zfs filesystem I understand the concern, but I'm not sure that this is the best way of achieving your goal. What would you want to observe if your system hit the upper limit in zfs_max_phys_mem? cheers, James C. McPherson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Re: Query on ZFS
Jaganraj Janarthanan wrote: Customer had come back saying. I've tried this and it doesn't work. It still displays the size as 1 gig after saying it's bringing the lun back online. Eric Schrock wrote On 08/15/06 20:49,: [ For ZFS discussions, try 'zfs-discuss@opensolaris.org' or '[EMAIL PROTECTED]' if there is proprietary information ] ZFS will detect the expanded LUN when it re-opens the device. There's currently no way to automatically do this. However, if you do a 'zpool online pool device', this will have the side effect of reopening the device and should pick up the expanded size. If it doesn't, then please let us know. - Eric Hi Jaganraj, As I recall, one of the projects that's still on the table within the relevant team is lun expansion. I don't recall any email (prior to being RIFfed) saying that the project has reached integration yet. Until it does, you'll probably have to reboot :( cheers, James C. McPherson ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Apple Time Machine
On 8/7/06, Adam Leventhal [EMAIL PROTECTED] wrote: Needless to say, this was a pretty interesting piece of the keynote from a technical point of view that had quite a few of us scratching our heads. After talking to some Apple engineers, it seems like what they're doing is more or less this: When a file is modified, the kernel fires off an event which a user-land daemon listens for. Every so often, the user-land daemon does something like a snapshot of the affected portions of the filesystem with hard links (including hard links to directories -- I'm not making this up). That might be a bit off, but it's the impression I was left with. Anyhow, very slick UI, sort of dubious back end, interesting possibility for integration with ZFS. You were spot on with your description and conclusion. More details here: http://arstechnica.com/staff/fatbits.ars/2006/8/15/4995 Steve ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Boot Disk
Dick Davies [EMAIL PROTECTED] wrote: As an aside, is there a general method to generate bootable opensolaris DVDs? The only way I know of getting opensolaris on is installing sxcr and then BFUing on top. A year ago, I did publish a toolkit to create bootable SchilliX CDs/DVDs. Would this help? Jörg -- EMail:[EMAIL PROTECTED] (home) Jörg Schilling D-13353 Berlin [EMAIL PROTECTED](uni) [EMAIL PROTECTED] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool import - cannot mount [...] directory is not empty
Robert, Are you sure that nfs-s5-p0/d5110 and nfs-s5-p0/d5111 are mounted following the import? These messages imply that the d5110 and d5111 directories in the top-level filesystem of pool nfs-s5-p0 are not empty. Could you verify that 'df /nfs-s5-p0/d5110' displays nfs-s5-p0/d5110 as the Filesystem (and not just nfs-s5-p0)? -Mark Robert Milkowski wrote: All pools were exported than I tried to import one-by-one and got this with only a first pool. bash-3.00# zpool export nfs-s5-p4 nfs-s5-s5 nfs-s5-s6 nfs-s5-s7 nfs-s5-s8 bash-3.00# zpool import nfs-s5-p4 cannot mount '/nfs-s5-p4/d5139': directory is not empty cannot mount '/nfs-s5-p4/d5141': directory is not empty cannot mount '/nfs-s5-p4/d5138': directory is not empty cannot mount '/nfs-s5-p4/d5142': directory is not empty bash-3.00# df -h /nfs-s5-p4/d5139 Filesystem size used avail capacity Mounted on nfs-s5-p4/d5139600G 556G44G93%/nfs-s5-p4/d5139 bash-3.00# zpool export nfs-s5-p4 bash-3.00# ls -l /nfs-s5-p4/d5139 /nfs-s5-p4/d5139: No such file or directory bash-3.00# ls -l /nfs-s5-p4/ total 0 bash-3.00# zpool import nfs-s5-p4 bash-3.00# uname -a SunOS XXX 5.11 snv_43 sun4u sparc SUNW,Sun-Fire-V240 bash-3.00# No problem with other pools - all other pools imported without any warnings. The same on another server (all pools were exported first): bash-3.00# zpool import nfs-s5-p0 cannot mount '/nfs-s5-p0/d5110': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag cannot mount 'nfs-s5-p0/d5112': mountpoint or dataset is busy cannot mount '/nfs-s5-p0/d5111': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag bash-3.00# zpool export nfs-s5-p0 bash-3.00# zpool import nfs-s5-p0 cannot mount '/nfs-s5-p0/d5110': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag cannot mount '/nfs-s5-p0/d5111': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag bash-3.00# zpool export nfs-s5-p0 bash-3.00# ls -la /nfs-s5-p0/ total 4 drwxr-xr-x 2 root other512 Jun 14 14:37 . drwxr-xr-x 40 root root1024 Aug 8 11:00 .. bash-3.00# zpool import nfs-s5-p0 cannot mount '/nfs-s5-p0/d5110': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag cannot mount 'nfs-s5-p0/d5112': mountpoint or dataset is busy cannot mount '/nfs-s5-p0/d5111': directory is not empty use legacy mountpoint to allow this behavior, or use the -O flag bash-3.00# bash-3.00# uname -a SunOS X 5.11 snv_39 sun4v sparc SUNW,Sun-Fire-T200 bash-3.00# All filesystems from that pool were however mounted. No problem with other pools - all other pools imported without any warnings. All filesystems in a pool have set sharenfs (actually sharenfs is set on a pool and then inherited by filesystems). Additionally nfs/server was disabled just before I exported pools and automatically started when first pool was imported. I belive there's already open bug for this. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re[2]: [zfs-discuss] zpool import - cannot mount [...] directory is not empty
Hello Mark, Wednesday, August 16, 2006, 3:23:43 PM, you wrote: MM Robert, MM Are you sure that nfs-s5-p0/d5110 and nfs-s5-p0/d5111 are mounted MM following the import? These messages imply that the d5110 and d5111 MM directories in the top-level filesystem of pool nfs-s5-p0 are not MM empty. Could you verify that 'df /nfs-s5-p0/d5110' displays MM nfs-s5-p0/d5110 as the Filesystem (and not just nfs-s5-p0)? They are definitely mounted and not empty. And df reports them as mounted file systems. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool import - cannot mount [...] directory is not empty
What does 'zfs list -o name,mountpoint' and 'zfs mount' show after the import? My only guess is that you have some explicit mountpoint set that's confusing the DSl-orderered mounting code. If this is the case, this was fixed in build 46 (likely to be in S10u4) to always mount datasets in mountpoint order, regardless of their hierarchical relationship. However, you may be hitting some other bug that we haven't seen before. - Eric On Wed, Aug 16, 2006 at 03:32:08PM +0200, Robert Milkowski wrote: Hello Mark, Wednesday, August 16, 2006, 3:23:43 PM, you wrote: MM Robert, MM Are you sure that nfs-s5-p0/d5110 and nfs-s5-p0/d5111 are mounted MM following the import? These messages imply that the d5110 and d5111 MM directories in the top-level filesystem of pool nfs-s5-p0 are not MM empty. Could you verify that 'df /nfs-s5-p0/d5110' displays MM nfs-s5-p0/d5110 as the Filesystem (and not just nfs-s5-p0)? They are definitely mounted and not empty. And df reports them as mounted file systems. -- Best regards, Robertmailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zpool import/export
This seems like a reasonable RFE. Feel free to file it at bugs.opensolaris.org. - Eric On Wed, Aug 16, 2006 at 06:44:44AM +0200, Robert Milkowski wrote: Hello zfs-discuss, I do have several pools in a SAN shared environment where some pools are mounted by one server and some by another. Now I can do 'zpool export A B C D' But I can't do 'zpool import A B C D' import -a isn't an option since I want import only those pools I just exported. It's not a big deal but it's annoying I can't import these pool with one command line like I can export them. Ideally I should just go to shell history and change export to import and voila! -- Best regards, Robert mailto:[EMAIL PROTECTED] http://milek.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- Eric Schrock, Solaris Kernel Development http://blogs.sun.com/eschrock ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Information on ZFS API?
I'm having trouble finding information on any hooks into ZFS. Is there information on a ZFS API so I can access ZFS information directly as opposed to having to constantly parse 'zpool' and 'zfs' command output? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Information on ZFS API?
On Wed, 2006-08-16 at 11:49 -0400, Eric Enright wrote: On 8/16/06, William Fretts-Saxton [EMAIL PROTECTED] wrote: I'm having trouble finding information on any hooks into ZFS. Is there information on a ZFS API so I can access ZFS information directly as opposed to having to constantly parse 'zpool' and 'zfs' command output? libzfs: http://cvs.opensolaris.org/source/xref/on/usr/src/lib/libzfs/common/ Careful -- this isn't an API -- it's part of the implementation of ZFS, and as a result it is subject to incompatible change at any time without advanced warning. In terms of our Interface Taxonomy, this is considered Project Private. If you have specific requirements for a programmatic interface to ZFS, let's hear them... it's not uncommon for Private interfaces to get put through some renovation and housecleaning and then promoted to a public interface, but it's best for the conversation to happen first... (And I can't speak for the plans/intentions of the core zfs team on this..) - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Information on ZFS API?
This is my first post on the opensolaris.org forums. I'm still trying to figure out whether I'm actually on the zfs-discuss alias and, if not, how to add myself to it! Anyway, I'm looking for a way to get information like zpool list or zfs get all filesystem through an API instead of parsing the information from the CLI. In the ideal world, I could just get a zpool object from the OS, which contained the zpool information and various zfs objects and do whatever I wanted with them. Bill Sommerfeld wrote: On Wed, 2006-08-16 at 11:49 -0400, Eric Enright wrote: On 8/16/06, William Fretts-Saxton [EMAIL PROTECTED] wrote: I'm having trouble finding information on any hooks into ZFS. Is there information on a ZFS API so I can access ZFS information directly as opposed to having to constantly parse 'zpool' and 'zfs' command output? libzfs: http://cvs.opensolaris.org/source/xref/on/usr/src/lib/libzfs/common/ Careful -- this isn't an API -- it's part of the implementation of ZFS, and as a result it is subject to incompatible change at any time without advanced warning. In terms of our Interface Taxonomy, this is considered Project Private. If you have specific requirements for a programmatic interface to ZFS, let's hear them... it's not uncommon for Private interfaces to get put through some renovation and housecleaning and then promoted to a public interface, but it's best for the conversation to happen first... (And I can't speak for the plans/intentions of the core zfs team on this..) - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re: [zfs-discuss] multi-layer ZFS filesystems and exporting: my stupid question for the day
On 8/16/06, Frank Cusack [EMAIL PROTECTED] wrote: On August 16, 2006 10:25:18 AM -0700 Joe Little [EMAIL PROTECTED] wrote: Is there a way to allow simple export commands the traverse multiple ZFS filesystems for exporting? I'd hate to have to have hundreds of mounts required for every point in a given tree (we have users, projects, src, etc) Set the sharenfs property on the filesystems and use the automounter on the client. Damn. We are hoping to move away from automounters and maintenance of such (we use NeoPath, for example, for virtual aggregation of the paths). In the NAS world, sometimes automounts are not available. So, if this is true, you'll likely need to count me as one of those people who says we can't make a filesystem per user, and give us user quotas now please :) -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] neopath vs automount
Sorry, I'm an email deleter, not a hoarder, so this is a new thread. Usually I save a thread I'm interested in for awhile before killing it, but I jumped the gun this time. Anyway ... I looked up neopath, cool product! But ISTM that to use it would give up some zfs features like snapshots. I would think that for home dirs, you can easily get away with using the automounter and you wouldn't give up a lot vs neopath functionality. -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: Re: Re: [zfs-discuss] multi-layer ZFS filesystems and exporting: my stupid question for the day
On 8/16/06, Frank Cusack [EMAIL PROTECTED] wrote: On August 16, 2006 10:34:31 AM -0700 Joe Little [EMAIL PROTECTED] wrote: On 8/16/06, Frank Cusack [EMAIL PROTECTED] wrote: On August 16, 2006 10:25:18 AM -0700 Joe Little [EMAIL PROTECTED] wrote: Is there a way to allow simple export commands the traverse multiple ZFS filesystems for exporting? I'd hate to have to have hundreds of mounts required for every point in a given tree (we have users, projects, src, etc) Set the sharenfs property on the filesystems and use the automounter on the client. Damn. We are hoping to move away from automounters and maintenance of such (we use NeoPath, for example, for virtual aggregation of the paths). I don't know what NeoPath is but automounts are trivial or at least easy to maintain even for very large sites. You are using path wildcards and DNS aliases, yes? used to.. running away quickly. For different *nix flavors, pick one of autofs, automountd, etc.. Nohide support, no nohide support, netgroups, LDAP, it all gets ugly pretty quickly. We want to move to statically definied mount trees similar to AFS managed centrally for all using NFS as the common protocol (aka the NeoPath) In the NAS world, sometimes automounts are not available. So, if this is true, you'll likely need to count me as one of those people who says we can't make a filesystem per user, and give us user quotas now please :) I don't understand. If an automount is not available, how is that different than the nfs server itself not being available. Or do you mean some clients do not have an automounter. Some clients don't have automounters available. Also some servers: Example is gateways/proxys (think SMB proxy w/o automounter support). Other clients in heavy use require lots of changes to get what you propose, such as OSX's automounter (storing the results in NetInfo) -frank ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS write performance problem with compression set to ON
Test setup: - E2900 with 12 US-IV+ 1.5GHz processor, 96GB memory, 2x2Gbps FC HBAs, MPxIO in round-robbin config. - 50x64GB EMC disks presented on both 2 FCs. - ZFS pool defined using all 50 disks - Multiple ZFS filesystems built on the above pool. I'm observing the following: - When the filesystems have compress=OFF and I do bulk reads/writes (8 parallel 'cp's running between ZFS filesystems) I observe approximately 200-250MB/S consolidated I/O; writes in the 100MB/S range. I get these numbers running 'zpool iostat 5'. I see the same read/write ratio for the duration of the test. - When the filesystems have compress=ON I see the following: reads from compressed filesystems come in waves; zpool will report for long durations (60+ seconds) no read activity while the write activity is consistently reported at 20MB/S (no variation in the write rate throughtout the test.) - The machine is mostly idling during the entire test; both cases. - ZFS reports 4:1 compresson ratio for my filesystem. I'm puzzled by the following: - Why do reads comes in waves with compression=ON; it almost feels like ZFS reads a bunch of data and then proceeds to compress it before writing it out. This tells me there is not a read bottleneck; meaning there is no starvation of the compress routine due to the following facts: CPU/Machine/IO is not saturated in any shape or form. - Why then does ZFS generate substantially lower write throughput (magical 20MB/S spread evenly across the 50 disks, 0.4MB/S each)? Can anybody shed any light on this anomoloy (?). Mr. Bonwick I hope you're reading this post. BTW, we love the ZFS and are looking forward to rolling out aggresively in our new project. I'd like to take advantage of the compression since we're mostly I/O bound and we've plenty of CPU/Memory. Thanks. This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Re: system unresponsive after issuing a zpool attach
I have similar behaviour on S10 U2 but in a different situation. I had working mirror with one of mirrors failed: mirror DEGRADED 0 0 0 c0d0 ONLINE 0 0 0 c0d1 UNAVAILABLE 0 0 0 After replacing corrupted hard disk i've run: # zpool replace tank c0d1 And it started replacement/resilvering... after few minutes system became unavailbale. Reboot only gives me a few minutes, then resilvering make system unresponsible. Is there any workaroud or patch for this problem??? This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
