[issue47072] Database corruption with the shelve module

2022-03-27 Thread Hubert Tournier
Hubert Tournier added the comment: The storage format used under Windows is completely different from the one used under Unix (or *BSD). Apart from the .dat datafile, there is a .dir index file with CSV lines such as "'key', (offset, length)". Whereas under Unix (or *B

[issue47072] Database corruption with the shelve module

2022-03-27 Thread Hubert Tournier
Hubert Tournier added the comment: Additional note: the test code WORKS under Windows 8.1 / Python 3.9.1 (though the data file is suffixed .dat instead of .db) resulting in a 4 MB database with 1065 records, some of them > 11 KB. So maybe the bug is system dependent. -- compone

[issue47072] Database corruption with the shelve module

2022-03-26 Thread Hubert Tournier
Hubert Tournier added the comment: I modified the test program to better reflect the size of the data structures stored in shelve (sys.getsizeof() which I used was far off the real size). I saw that the database was corrupted with big records, though even bigger previous records had not

[issue47072] Database corruption with the shelve module

2022-03-26 Thread Hubert Tournier
Hubert Tournier added the comment: Hello, Same results with Python 3.10.4: [...] Adding 185.220.102.6 Database has 62 records for 442368 bytes. Last record was 640 bytes long Traceback (most recent call last): File "./shelve-test.py", line 84, in _verify_whois_cache() File

[issue47072] Database corruption with the shelve module

2022-03-20 Thread Hubert Tournier
New submission from Hubert Tournier : After adding a few records, the shelve module corrupts the database keys (the database is still readable if an element key is known, but no more iterable): Traceback (most recent call last): File "./shelve-test.py", line 81, in _verify_w