On 9/24/07, Paul B. Henson <[EMAIL PROTECTED]> wrote: > On Sat, 22 Sep 2007, Peter Tribble wrote: > > > filesystem per user on the server, just to see how it would work. While > > managing 20,00 filesystems with the automounter was trivial, the attempt > > to manage 20,000 zfs filesystems wasn't entirely successful. In fact, > > based on that experience I simply wouldn't go down the road of one user > > per filesystem. > > Really? Could you provide further detail about what problems you > experienced? Our current filesystem based on DFS effectively utilizes a > separate filesystem per user (although in DFS terminology they are called > filesets), and we've never had a problem managing them.
This was some time ago (a very long time ago, actually). There are two fundamental problems: 1. Each zfs filesystem consumes kernel memory. Significant amounts, 64K is what we worked out at the time. For normal numbers of filesystems that's not a problem; multiply it by tens of thousands and you start to hit serious resource usage. 2. The zfs utilities didn't scale well as the number of filesystems increased. I just kept on issuing zfs create until the machine had had enough. It got through the first 10,000 without too much difficulty (as I recall that took several hours), but soon got bogged down after that, to the point where it took a day to do anything. At which point (at about 15000 filesystems on a 1G system) it ran out of kernel memory and died. At this point it wouldn't even boot. I know that some work has gone into improving the performance of the utilities, and things like in-kernel sharetab (we never even tried to share all those filesystems) are there to improve scalability. Perhaps I should find a spare machine and try repeating the experiment. -- -Peter Tribble http://www.petertribble.co.uk/ - http://ptribble.blogspot.com/ _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
