Re: interesting limitation
On Mon, 31 Mar 2003, Dave O wrote: 2 level hashing would work, but I don't know if Cyrus supports that. It would most likely be trivial to implement. eg spool/s/sm/user/smith spool/s/m/user/smith? ian
Re: interesting limitation
2 level hashing would work, but I don't know if Cyrus supports that. It would most likely be trivial to implement. eg spool/s/sm/user/smith On Sat, 29 Mar 2003, Jure Pecar wrote: Hi all, Recently i was testing a 2.2 branch on linux with Veritas vxfs. I wanted to create 20 users in the form of userN, where n is 1..20. I soon found out that vxfs won't let me create more than 32k subdirs in one dir. This is clearly a limitation of the filesystem. How does other filesystems handle this? The solution here is full dir hash. But, the next limit is at 26*32k users. Is anyone actually nearing this number of users on a single box? Probably not, but who knows what the future may bring ... -- Jure Pecar
Re: interesting limitation
On Mon, 31 Mar 2003 10:42:39 -0500 (EST) Dave O [EMAIL PROTECTED] wrote: 2 level hashing would work, but I don't know if Cyrus supports that. It would most likely be trivial to implement. eg spool/s/sm/user/smith Yes, i was thinking about that too ... In fact i would prefer it over fulldir hash code, because this way i always know where on disk the user's mailbox is. -- Jure Pecar
Re: interesting limitation
Dave O wrote: 2 level hashing would work, but I don't know if Cyrus supports that. It would most likely be trivial to implement. eg spool/s/sm/user/smith Or in the case of full dir hashing, have a second hash function and hash the names that get assigned to one bin into an additional set of bins. As was previously mentioned, having multiple partitions also solves this problem but since the trend seems to be consolidating storage it would be nice to be able to handle the large numbers of users in a single partition rather than creating multiple partitions in the same filesystem with the associated administrative hassle. -- John A. Tamplin Unix System Administrator Emory University, School of Public Health +1 404/727-9931
interesting limitation
Hi all, Recently i was testing a 2.2 branch on linux with Veritas vxfs. I wanted to create 20 users in the form of userN, where n is 1..20. I soon found out that vxfs won't let me create more than 32k subdirs in one dir. This is clearly a limitation of the filesystem. How does other filesystems handle this? The solution here is full dir hash. But, the next limit is at 26*32k users. Is anyone actually nearing this number of users on a single box? Probably not, but who knows what the future may bring ... -- Jure Pecar
Re: interesting limitation
Hi! Yes, and ext2/3 has the same limitation. Ext2/3 code in the Linux kernel can be modified (one line) and recompiled to allow more directories, up to cca. 65000. ReiserFS is known not to have this limitation, if I'm right. But the solution should be a (configurable) more scalable hashing method, for a large user base. On Sat, 2003-03-29 at 20:04, Jure Pecar wrote: Hi all, Recently i was testing a 2.2 branch on linux with Veritas vxfs. I wanted to create 20 users in the form of userN, where n is 1..20. I soon found out that vxfs won't let me create more than 32k subdirs in one dir. This is clearly a limitation of the filesystem. How does other filesystems handle this? The solution here is full dir hash. But, the next limit is at 26*32k users. Is anyone actually nearing this number of users on a single box? Probably not, but who knows what the future may bring ... -- | Noll Janos [EMAIL PROTECTED] | http://www.johnzero.hu | | Expect the unexpected!| ICQ# 4547866 | Be free! |
Re: interesting limitation
On Sat, 29 Mar 2003, Jure Pecar wrote: This is clearly a limitation of the filesystem. How does other filesystems handle this? I have a XFS dir with more than 40k objects, and it still performs okay... but I had some not-so-nice words with the people involved and told them to learn about hash directories, and to fix that :) The solution here is full dir hash. But, the next limit is at 26*32k users. Should be enough users for a single spool, don't ya think? Cyrus Murder might fix your problem, though, if you don't mind multiple imapd instances (in multiple boxes if at all possible). -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh
Re: interesting limitation
On Sat, 29 Mar 2003, Henrique de Moraes Holschuh wrote: Should be enough users for a single spool, don't ya think? Cyrus Murder might fix your problem, though, if you don't mind multiple imapd instances (in multiple boxes if at all possible). If the problem is just number of users on one box, you can just split between multiple partitions. There's no reason to add the complexity of a murder. Of course, even if it is too many for one box, you should look at perdition or another IMAP proxy first. -Rob -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Rob Siemborski * Andrew Systems Group * Cyert Hall 207 * 412-268-7456 Research Systems Programmer * /usr/contributed Gatekeeper
Re: interesting limitation
On Sat, 29 Mar 2003, Jure Pecar wrote: Hi all, Recently i was testing a 2.2 branch on linux with Veritas vxfs. I wanted to create 20 users in the form of userN, where n is 1..20. I soon found out that vxfs won't let me create more than 32k subdirs in one dir. This is clearly a limitation of the filesystem. How does other filesystems handle this? The solution here is full dir hash. But, the next limit is at 26*32k users. Is anyone actually nearing this number of users on a single box? Probably not, but who knows what the future may bring ... Add more partitions. -- Igor
