On Fri, Sep 02, 2011 at 06:30:57AM -0500, Terry Cumaranatunge scratched on the wall: > Hello, > > We have an application that creates many small databases (over 100K) to be > able to control the size of the database and provide more deterministic > performance. At process startup, it opens many of these databases in a loop > to keep them open for faster transaction response times. The behavior we are > noticing is that the it takes progressively a longer time for each > sqlite3_open_v2 to complete as the number of databases kept > opened increases. These are some of the measurements: > > 5000 DBs = 3 secs > 10000 DBs = 11 secs > 15000 DBs = 35 secs > 20000 DBs = 75 secs > > Many processes can concurrently open 5000 db's at the same time and it takes > about the same time as a single process doing the work. So, it doesn't > appear to be related to the OS related issue with the number of opens.
I'd guess the issue is OS related, but is at the process level, not the system level. For example, if a process holds open file descriptors in an array, many operations (such as scanning for the lowest unused descriptor number) are going to be O(n). I would run a quick test that just calls the system level open(2) type call, and see if you observe the same type of slow-down. -j -- Jay A. Kreibich < J A Y @ K R E I B I.C H > "Intelligence is like underwear: it is important that you have it, but showing it to the wrong people has the tendency to make them feel uncomfortable." -- Angela Johnson _______________________________________________ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
