On Sat, Mar 10, 2012 at 10:02:11PM +0000, David Cantrell wrote: > Using DBI::Shell, executing this query against the database at > http://devel.cpantesters.org/cpanstats.db.gz ... > > SELECT id, state, tester, dist, version, platform, perl, osname > FROM cpanstats > WHERE id > 20618074 AND > state IN ('pass', 'fail', 'na', 'unknown') AND > perl != '0' > > gives ... > > id,state,tester,dist,version,platform,perl,osname > 20618075,'pass','r...@klanker.bingosnet.co.uk ((Charlie > &))','Net-Inspect','0.09','x86_64-linux','5.15.7','linux' > DBD::SQLite::st fetchrow_arrayref failed: database disk image is > malformed, <FIN> line 5. > > And it's not just because my download of the database aborted half way, > or my filesystem filling up, or it not being able to cope with big > files. > > This has been happening for a couple of days now - anyone know what's > going on? Barbie?
Andreas has already alerted me to this, but even recreating the SQLite DB from scratch has the same problem. The only thing I can think of is that the DB is so big now that SQLite has problems storing the data. This is why I want to look at an alternative method of getting at the data. I'm going to look at creating an API that returns the records in CSV or JSON format for a given range of IDs. This will allow you to maintain your own DB as you wish, rather than have to continually download several Gig repeatedly. Cheers, Barbie. -- Birmingham Perl Mongers <http://birmingham.pm.org> Memoirs Of A Roadie <http://barbie.missbarbell.co.uk> CPAN Testers Blog <http://blog.cpantesters.org> YAPC Conference Surveys <http://yapc-surveys.org> Ark Appreciation Pages <http://ark.eology.org>
