Re: [zfs-discuss] ZFS Group Quotas
Matt, > If the filesystem is old (created on pool version < 15), then the first time > you do "zfs userspace " or "zfs groupspace ", it will take some time > to do an initial gather of the accounting information. This is good to know since we do have some upgraded filesystems / pools, thanks! I noticed you mentioned the pool version, I assume this is because a zfs version 4 filesystem can only exist on a zpool version 15 or later. Does that first run of zfs userspace or zfs groupspace gather all accounting information whether or not quotas have been set for all users or groups, or will there be an "full" accounting scan everytime a new quota is set and zfs groupspace is run? Thanks, Jordan ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
Jordan Schwartz wrote: ZFSfolk, Pardon the slightly offtopic post, but I figured this would be a good forum to get some feedback. I am looking at implementing zfs group quotas on some X4540s and X4140/J4400s, 64GB of RAM per server, running Solaris 10 Update 8 servers with IDR143158-06. There is one large filesystem per server that is served via NFSv3 to linux based clients for web and email loads. There will be at least a few thousand group quotas per filesystem. Are there any scaling/performance issues with group based quotas? No, this should work just fine. For the filesystems that are already populated with thousands of groups and terabytes of data in relatively small files, will there be any performance impacts as the quotas are created? If the filesystem is old (created on pool version < 15), then the first time you do "zfs userspace " or "zfs groupspace ", it will take some time to do an initial gather of the accounting information. After that happens, there should be no noticeable performance impact of setting or using user or group quotas. Also for the pre-populated filesystems will "zfs get groupsp...@$gid $zpool/$fs" return the total usage for the group? Yes. (Though if the filesystem is old you may need to do "zfs groupspace" once to gather the data.) --matt ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
Well I seemed to have hit on that hot button topic of NFSv4, (good thing I didn't mention that we are running IPv4). To get back to the topic, is anyone running ZFS group quota on large filesystem with lots of smaller files and thousands for groups per filesystem, or have any quota related experinces to share? Thanks, Jordan On Tue, Aug 17, 2010 at 5:20 PM, Jordan Schwartz wrote: > ZFSfolk, > > Pardon the slightly offtopic post, but I figured this would be a good > forum to get some feedback. > > I am looking at implementing zfs group quotas on some X4540s and > X4140/J4400s, 64GB of RAM per server, running Solaris 10 Update 8 > servers with IDR143158-06. > > There is one large filesystem per server that is served via NFSv3 to > linux based clients for web and email loads. There will be at least a > few thousand group quotas per filesystem. > > Are there any scaling/performance issues with group based quotas? > > For the filesystems that are already populated with thousands of > groups and terabytes of data in relatively small files, will there be > any performance impacts as the quotas are created? > > Also for the pre-populated filesystems will "zfs get groupsp...@$gid > $zpool/$fs" return the total usage for the group? > > Thanks for any feedback, > > Jordan Schwartz > ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
> > Also the linux NFSv4 client is bugged (as in hang-the-whole-machine bugged). > I am deploying a new osol fileserver for home directories and I'm using NFSv3 > + automounter (because I am also using one dataset per user, and thus I have > to mount each home dir separately). We are also in the same boat here. I have about 125TB of ZFS storage in production currently, running OSOL, across 5 X4540s. We tried the NFSv4 route, and crawled back to NFSv3 and the linux automounter because NFSv4 on Linux is *that* broken. As in hung-disk-io-that-wedges-the-whole-box broken. We know that NFSv3 was never meant for the scale we're using it at, but we have no choice in the matter. On the topic of Linux clients, NFS and ZFS: We've also found that Linux is bad at handling lots of mounts/umounts. We will occasionally find a client where the automounter requested a mount, but it never actually completed. It'll show as mounted in /proc/mounts, but won't *actually* be mounted. A umount -f for the affected filesystem fixes this. On ~250 clients in an HPC environment, we'll see such an error every week or so. I'm hoping that recent versions of linux (i.e. RHEL 6) are a bit better at NFSv4, but i'm not holding my breath. -- Greg Mason HPC Administrator Michigan State University Institute for Cyber Enabled Research High Performance Computing Center web: www.icer.msu.edu email: gma...@msu.edu ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
Il giorno 18/ago/2010, alle ore 21.24, David Magda ha scritto: > On Wed, August 18, 2010 15:14, Linder, Doug wrote: >> I've noticed that everytime someone mentions using NFS with ZFS here, they >> always seem to be using NFSv3. Is there a reason for this that I just >> don't know about? > At $WORK it's generally namespace issues: > > http://blogs.sun.com/tdh/entry/linux_nfsv4_namespace_implementation_fools Also the linux NFSv4 client is bugged (as in hang-the-whole-machine bugged). I am deploying a new osol fileserver for home directories and I'm using NFSv3 + automounter (because I am also using one dataset per user, and thus I have to mount each home dir separately). -- Simone Caldana ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
On Wed, August 18, 2010 15:14, Linder, Doug wrote: > I've noticed that everytime someone mentions using NFS with ZFS here, they > always seem to be using NFSv3. Is there a reason for this that I just > don't know about? To me, using NFSv4 is a no-brainer. ZFS supports it > natively, it supports all the wonderful extra capabilities that the ZFS > ACLs allow, has stronger security, stateful protocol, and all kinds of > other nifty stuff. Why do people seem to be clinging so rabidly to the > old version? Is there some technical reason I'm missing? At $WORK it's generally namespace issues: http://blogs.sun.com/tdh/entry/linux_nfsv4_namespace_implementation_fools Haven't really found a use for the "extras" that NFSv4 adds, so it's not worth the effort. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS Group Quotas
Jordan Schwartz wrote: > There is one large filesystem per server that is served via NFSv3 to I've noticed that everytime someone mentions using NFS with ZFS here, they always seem to be using NFSv3. Is there a reason for this that I just don't know about? To me, using NFSv4 is a no-brainer. ZFS supports it natively, it supports all the wonderful extra capabilities that the ZFS ACLs allow, has stronger security, stateful protocol, and all kinds of other nifty stuff. Why do people seem to be clinging so rabidly to the old version? Is there some technical reason I'm missing? Doug Linder I apologize for all the stupid cruft below that my company's mail server adds. -- Learn more about Merchant Link at www.merchantlink.com. THIS MESSAGE IS CONFIDENTIAL. This e-mail message and any attachments are proprietary and confidential information intended only for the use of the recipient(s) named above. If you are not the intended recipient, you may not print, distribute, or copy this message or any attachments. If you have received this communication in error, please notify the sender by return e-mail and delete this message and any attachments from your computer. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS Group Quotas
ZFSfolk, Pardon the slightly offtopic post, but I figured this would be a good forum to get some feedback. I am looking at implementing zfs group quotas on some X4540s and X4140/J4400s, 64GB of RAM per server, running Solaris 10 Update 8 servers with IDR143158-06. There is one large filesystem per server that is served via NFSv3 to linux based clients for web and email loads. There will be at least a few thousand group quotas per filesystem. Are there any scaling/performance issues with group based quotas? For the filesystems that are already populated with thousands of groups and terabytes of data in relatively small files, will there be any performance impacts as the quotas are created? Also for the pre-populated filesystems will "zfs get groupsp...@$gid $zpool/$fs" return the total usage for the group? Thanks for any feedback, Jordan Schwartz ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
