Found the problem with some help from a friend in the Chicago Python users group.
One problem is flushing before unlocking the file. Another is that opening the file for writing truncates the file before the exclusive lock is acquired. On Feb 29, 11:46 am, Massimo Di Pierro <massimo.dipie...@gmail.com> wrote: > I believe we have a problem with file locking. I managed to isolate > the problem in this non-web2py pure-python program: > > ==== begin file test.py ====== > import fcntl > import multiprocessing > import unittest > > N=10000 > > def lock(fp, mode): > fcntl.flock(fp.fileno(), mode) > > def unlock(fp): > fcntl.flock(fp.fileno(), fcntl.LOCK_UN) > > def read_write(args): > (filename, iterations) = args > for i in range(0, iterations): > fp = open(filename,'r') > lock(fp,fcntl.LOCK_SH) # shared > lock > content = fp.read() > unlock(fp) # > unlock > fp.close() > if len(content)!=N: > return False > fp = open(filename,'w') > lock(fp,fcntl.LOCK_EX) # exclusive > lock > fp.write(content) > unlock(fp) # > unlock > fp.close() > return True > > class TestParallelLocks(unittest.TestCase): > > def setUp(self): > self.filename = 'test.txt' > contents = 'x'*N > fp = open(self.filename,'w') > fp.write(contents) > fp.close() > > def tearDown(self): > try: > os.remove(self.filename) > except: > pass > > def test_reads_and_writes(self): > readwriters = 10 > pool = multiprocessing.Pool(processes = readwriters) > results = pool.map(read_write, [[self.filename, 10]] * > readwriters) > for result in results: > self.assertTrue(result) > > if __name__ == '__main__': > unittest.main() > ===== end file test.py ==== > > If I run it with python test.py it will often fail the test (tested on > Mac Leopard, Lion, Ubuntu, Python 2.5, 2.6, 2.7). > > HELP! Can you reproduce the problem? Do you see anything wrong with > the code? > > Massimo
