I'm testing new code, and my latest trial run ended with a segmentation fault after about 5 hours. I'm running Python 3.5 and its standard sqlite3 module On Xubuntu 16.04.3 LTS. The code is short -- about 300 lines.
This particular program is merging two databases. The result has reached 25 GB, roughly 1/3 of what I expect of the final result (over 100M rows). The filesystem is a RAID with 2+ TB free. The machine is a Core i7 with 32 GB RAM and 0 swap has been used since the last reboot. Nothing else much is running on this machine except some idle terminal and browser windows. Here's my prime suspect: I'm using WAL, and the journal is 543 MB. I hadn't given it much thought, but could this be more than the software really wants to deal with? I'm going to try doing occasional commits (every 100K inserts/updates perhaps,) but I'd like some help: 1. If I'm on the right track, tell me so I can stop worrying and proceed with development. 2. If I'm on the wrong track, help me figure out how to debug the problem. I can probably find out what particular part of the merge it had reached, but it's going to take quite a while. I'm pretty good with GDB but I have no idea how to explore a running Python program. The project is a hobby, so there's nothing proprietary, and I can post any information that would help. ++ kevin -- word of the year: *kakistocracy* _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
