Re: Max files in unix folder from PIL process

2005-03-29 Thread Ivan Van Laningham
Hi All-- Rowdy wrote: FreeDB (CD database) stores one file per CD in one directory per category. The misc category/directory on my FreeBSD 5.3 system currently contains around 481,571 small files. The rock directory/category contains 449,208 files. As some have said, ls is *very* slow

Re: Max files in unix folder from PIL process

2005-03-29 Thread Kane
Yes I'm talking Linux not BSD so with any luck you won't have the same 'ls' issue; it is not a crash but painfully slow. The only other issue I recall is wildcards fail if they encompass too many files (presumably a bash/max command line size). I would expect the various GUI file managers may

Max files in unix folder from PIL process

2005-03-28 Thread David Pratt
Hi. I am creating a python application that uses PIL to generate thumbnails and sized images. It is beginning to look the volume of images will be large. This has got me to thinking. Is there a number that Unix can handle in a single directory. I am using FreeBSD4.x at the moment. I am

Re: Max files in unix folder from PIL process

2005-03-28 Thread Kane
I ran into a similar situation with a massive directory of PIL generated images (around 10k). No problems on the filesystem/Python side of things but other tools (most noteably 'ls') don't cope very well.As it happens my data has natural groups so I broke the big dir into subdirs to sidestep

Re: Max files in unix folder from PIL process

2005-03-28 Thread David Pratt
Hi Jason. Many thanks your reply. This is good to know about ls - what did it do? Was it just slow or did the server or machine die? My images will be going into the path of a web server. This is unchartered territory for me and I don't know whether there will be speed and access problems

Re: Max files in unix folder from PIL process

2005-03-28 Thread Andrew Dalke
Kane wrote: I ran into a similar situation with a massive directory of PIL generated images (around 10k). No problems on the filesystem/Python side of things but other tools (most noteably 'ls') don't cope very well. My experience suggests that 'ls' has a lousy sort routine or that it takes

Re: Max files in unix folder from PIL process

2005-03-28 Thread Martin v. Löwis
David Pratt wrote: Hi. I am creating a python application that uses PIL to generate thumbnails and sized images. It is beginning to look the volume of images will be large. This has got me to thinking. Is there a number that Unix can handle in a single directory. I am using FreeBSD4.x at the

Re: Max files in unix folder from PIL process

2005-03-28 Thread Rowdy
David Pratt wrote: Hi. I am creating a python application that uses PIL to generate thumbnails and sized images. It is beginning to look the volume of images will be large. This has got me to thinking. Is there a number that Unix can handle in a single directory. I am using FreeBSD4.x at the