2012/8/1 Steven D'Aprano <steve+comp.lang.pyt...@pearwood.info>: > On Wed, 01 Aug 2012 14:01:45 +0100, andrea crotti wrote: > >> Full traceback: >> >> Exception in thread Thread-8: > > "DANGER DANGER DANGER WILL ROBINSON!!!" > > Why didn't you say that there were threads involved? That puts a > completely different perspective on the problem. > > I *was* going to write back and say that you probably had either file > system corruption, or network errors. But now that I can see that you > have threads, I will revise that and say that you probably have a bug in > your thread handling code. > > I must say, Andrea, your initial post asking for help was EXTREMELY > misleading. You over-simplified the problem to the point that it no > longer has any connection to the reality of the code you are running. > Please don't send us on wild goose chases after bugs in code that you > aren't actually running. > > >> there seems to be no clear pattern and just randmoly fails. > > When you start using threads, you have to expect these sorts of > intermittent bugs unless you are very careful. > > My guess is that you have a bug where two threads read from the same file > at the same time. Since each read shares state (the position of the file > pointer), you're going to get corruption. Because it depends on timing > details of which threads do what at exactly which microsecond, the effect > might as well be random. > > Example: suppose the file contains three blocks A B and C, and a > checksum. Thread 8 starts reading the file, and gets block A and B. Then > thread 2 starts reading it as well, and gets half of block C. Thread 8 > gets the rest of block C, calculates the checksum, and it doesn't match. > > I recommend that you run a file system check on the remote disk. If it > passes, you can eliminate file system corruption. Also, run some network > diagnostics, to eliminate corruption introduced in the network layer. But > I expect that you won't find anything there, and the problem is a simple > thread bug. Simple, but really, really hard to find. > > Good luck. >
Thanks a lot, that makes a lot of sense.. I haven't given this detail before because I didn't write this code, and I forgot that there were threads involved completely, I'm just trying to help to fix this bug. Your explanation makes a lot of sense, but it's still surprising that even just reading files without ever writing them can cause troubles using threads :/ -- http://mail.python.org/mailman/listinfo/python-list