hi mrab.. to your question... no file is acquired by multiple processes. and yeah, i've thought of copying the files to a separate dir for each child/client process to work with.. and in all honesty, this is as fast as i can envision at this time...
-----Original Message----- From: python-list-bounces+bedouglas=earthlink....@python.org [mailto:python-list-bounces+bedouglas=earthlink....@python.org]on Behalf Of MRAB Sent: Sunday, March 01, 2009 8:04 AM To: python-list@python.org Subject: Re: file locking... bruce wrote: > Hi. > > Got a bit of a question/issue that I'm trying to resolve. I'm asking this of > a few groups so bear with me. > > I'm considering a situation where I have multiple processes running, and > each process is going to access a number of files in a dir. Each process > accesses a unique group of files, and then writes the group of files to > another dir. I can easily handle this by using a form of locking, where I > have the processes lock/read a file and only access the group of files in > the dir based on the open/free status of the lockfile. > > However, the issue with the approach is that it's somewhat synchronous. I'm > looking for something that might be more asynchronous/parallel, in that I'd > like to have multiple processes each access a unique group of files from the > given dir as fast as possible. > > So.. Any thoughts/pointers/comments would be greatly appreciated. Any > pointers to academic research, etc.. would be useful. > You say "each process accesses a unique group of files". Does this mean that no two processes access the same file? If yes, then why do you need locking? If no, then could you lock, move the files into a work folder, and then unlock? (Remember that moving a file on the same volume (disk) should be a quick renaming.) There could be one work folder per process, although that isn't necessary because each process would know (or be told) which of the files in the work folder belonged to it. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
