Hi, I have now sanitized the logic in the rest of the code to not require these flags anymore. Thus I got rid of the frequent updates to each record, that were an abomination in the first place.
So I am left with decent DB-Operations which SQLite can manage perfectly well. Happy now. Thanks for all the help and for fantastic software. That said I can elaborate a bit: > That is a tad over 36,000 b-tree nodes per second. What are your actual > performance requirements? There are no hard requirements. But a 10-minute query every now and then really is outside of scope :-). I have inherited this piece of software. It used a fixed-size array in memory, blasted 1:1 in binary format out to disc periodically, as a database. That was fine except for two problems that needed adressing: 1) The device is limited to 128MB of RAM. That is for kernel, application and data. The current DB was 40MB. The number of records is quickly growing with business and is projected to become a problem soon. Also because of the fixed size it would have to be decided on a new maximum size and about 500 systems in the field upgraded. Only to repeat when that new maximum size is too small again. 2) Updating changed records is slow even in RAM because there is no indexing whatsoever. A larger number of overall records means more updates per time frame, the device is unresponsive while updating, and this is starting to become a problem. 2) I could have fixed by adding an indexing scheme but 1) is inherently unfixable. Any solution requires a switch to a disk-based system and any such system is going to be slower than stuff-it-ALL-into-a-contiguous-block-of-RAM . That is perfectly acceptable as long as the slowdown is well constrained. And it is now. So it was a shootout between some system between a homebrew on-disc-structure plus indexing, something like Berkeley-DB or a SQL-Engine. I preferred the latter because: - It would radically simplify the application code. And it did: The new version is less than 10% the application-LoC as the old one and much much cleaner. It almost looks a designed piece of software now as opposed to a smoldering heap of, of, something. - Also it would give me easy access to the database for debugging. Having a commandline tool to browse, query and update the data (and not having to write it myself) is a real plus. - After initially having rolled out an update to change to the DB-based code changes to the format of the data become much easier to handle (There have been several cases in the past where string fields needed resizing and so on, don't ask, it's all very sad). I was planning to clean up the client code to that DB backend anyway. That is the part doing all the ridiculous updates. But I had planned to do so in a seperate cycle. It IS an extremely ugly codebase and still breaks whenever I look at it hard enough. But now that I upgraded DB-backend to SQLite and fixed the basic algorithms in the client-code together it is beginning to resemble a real database-application and already works much better. Ciao, MM _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
