Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-21 Thread Daniel Richard G.
On Sat, 2009 Sep 19 21:19+0100, Mark Hindley wrote: Thanks, very helpful. I think I have found this. Can you verify it is fixed with this patch I've been running the patched apt-cacher since Saturday evening in the same 3-minute testing cycle, and with access.log now over 100K lines,

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-19 Thread Mark Hindley
On Mon, Sep 14, 2009 at 06:02:26PM -0400, Daniel Richard G. wrote: On Sun, 2009 Sep 13 09:40+0100, Mark Hindley wrote: Probably without. I think this is another concurrency/locking issue, but I have obviously not identified it completely correctly yet. Okay. Attached is a 100k-line

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-13 Thread Mark Hindley
On Sun, Sep 13, 2009 at 01:32:35AM -0400, Daniel Richard G. wrote: On Fri, 2009 Sep 11 10:22+0100, Mark Hindley wrote: I thought I had worked out a locking error that was causing this, but obviously not. Could you remove these last patches and turn on debugging and send the error.log so I

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-12 Thread Daniel Richard G.
On Fri, 2009 Sep 11 10:22+0100, Mark Hindley wrote: I thought I had worked out a locking error that was causing this, but obviously not. Could you remove these last patches and turn on debugging and send the error.log so I can try to see why $cached_file is not getting initialised. As you

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-11 Thread Mark Hindley
On Thu, Sep 10, 2009 at 04:26:50PM -0400, Daniel Richard G. wrote: On Thu, 2009 Sep 10 11:43+0100, Mark Hindley wrote: Could you try this patch and see if it is fixed for you. Sorry, that was incomplete. You will also need this: I'm still seeing some uninitialized value warnings, as

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-10 Thread Mark Hindley
On Wed, Sep 02, 2009 at 12:22:00PM -0400, Daniel Richard G. wrote: After a few hundred cycles of testing, this patch is looking solid. Running with checksumming enabled, and debugging disabled, error.log and db.log remain at zero size. Oh, but there was one minor issue. I was previously

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-10 Thread Mark Hindley
On Thu, Sep 10, 2009 at 10:33:28AM +0100, Mark Hindley wrote: On Wed, Sep 02, 2009 at 12:22:00PM -0400, Daniel Richard G. wrote: After a few hundred cycles of testing, this patch is looking solid. Running with checksumming enabled, and debugging disabled, error.log and db.log remain at zero

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-10 Thread Daniel Richard G.
On Thu, 2009 Sep 10 11:43+0100, Mark Hindley wrote: Could you try this patch and see if it is fixed for you. Sorry, that was incomplete. You will also need this: I'm still seeing some uninitialized value warnings, as well as a new flock() warning cropping up in spades. Here's what error.log

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-06 Thread Mark Hindley
On Wed, Sep 02, 2009 at 12:22:00PM -0400, Daniel Richard G. wrote: After a few hundred cycles of testing, this patch is looking solid. Running with checksumming enabled, and debugging disabled, error.log and db.log remain at zero size. Good. I am just using NYTProf to see if I can track down

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-06 Thread Mark Hindley
On Wed, Sep 02, 2009 at 12:22:00PM -0400, Daniel Richard G. wrote: After a few hundred cycles of testing, this patch is looking solid. Running with checksumming enabled, and debugging disabled, error.log and db.log remain at zero size. Oh, but there was one minor issue. I was previously

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-06 Thread Daniel Richard G.
On Sun, 2009 Sep 6 10:58+0100, Mark Hindley wrote: Are the types of file that $cache_status is uninitialised for all package files, all index files or a mixture? I was only testing downloads of index files, so no word on whether the warning would have come up for package files as well. --

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-06 Thread Mark Hindley
On Sun, Sep 06, 2009 at 06:34:31AM -0400, Daniel Richard G. wrote: On Sun, 2009 Sep 6 10:58+0100, Mark Hindley wrote: Are the types of file that $cache_status is uninitialised for all package files, all index files or a mixture? I was only testing downloads of index files, so no word on

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-06 Thread Daniel Richard G.
On Sun, 2009 Sep 6 11:41+0100, Mark Hindley wrote: OK. Is it 1 type of index file? From the looks of it, it's all types: $ grep UNKNOWN access.log | sed 's/.*_//' | sort -u Packages.bz2 Release Release.gpg Sources.bz2 (UNKNOWN was what I was initializing $cache_status with.) -- To

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-02 Thread Daniel Richard G.
After a few hundred cycles of testing, this patch is looking solid. Running with checksumming enabled, and debugging disabled, error.log and db.log remain at zero size. Oh, but there was one minor issue. I was previously seeing this warning come up repeatedly in error.log: Tue Sep 1 18:10:46

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-01 Thread Mark Hindley
On Mon, Aug 31, 2009 at 03:19:41PM -0400, Daniel Richard G. wrote: On Fri, 2009 Aug 28 01:02+0100, Mark Hindley wrote: Actually, I think it should be fixed by not releasing the lock until the end of the fetcher process. I'm still seeing occasional bzip2 errors with this patch. (I have a

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-09-01 Thread Daniel Richard G.
On Tue, 2009 Sep 1 08:39+0100, Mark Hindley wrote: I was holding off, because I wanted this fixed in it! Ah, okay. From the look of the control messages, I thought you'd uploaded it already. I have rewritten the decompression code to use native perl modules rather than external command

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-08-31 Thread Daniel Richard G.
On Fri, 2009 Aug 28 01:02+0100, Mark Hindley wrote: Actually, I think it should be fixed by not releasing the lock until the end of the fetcher process. I'm still seeing occasional bzip2 errors with this patch. (I have a good test rig for this now: four clients apt-get update-ing from a server

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-08-27 Thread Mark Hindley
On Wed, Aug 26, 2009 at 03:16:39PM -0400, Daniel Richard G. wrote: Package: apt-cacher Version: 1.6.8 I am testing apt-cacher with checksumming enabled. I start the server with an empty cache directory tree and no database file (such that the database has to be created from scratch), and

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-08-27 Thread Mark Hindley
On Thu, Aug 27, 2009 at 08:31:06PM +0100, Mark Hindley wrote: Could you test with this patch Too quick The patch is not right, because the $$pkfdref can be compressed and needs decompressing! However, I hope it will make the error disappear. M -- To UNSUBSCRIBE, email to

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-08-27 Thread Mark Hindley
On Thu, Aug 27, 2009 at 08:58:37PM +0100, Mark Hindley wrote: On Thu, Aug 27, 2009 at 08:31:06PM +0100, Mark Hindley wrote: Could you test with this patch Too quick The patch is not right, because the $$pkfdref can be compressed and needs decompressing! Actually, I think it should

Bug#543760: Checksumming leads to bzcat/bzip2 errors

2009-08-26 Thread Daniel Richard G.
Package: apt-cacher Version: 1.6.8 I am testing apt-cacher with checksumming enabled. I start the server with an empty cache directory tree and no database file (such that the database has to be created from scratch), and then run apt-get update on two client machines (in this case, an Ubuntu