On Fri, Aug 28, 2009 at 12:39:04AM -0500, Shawn Walker wrote: > Nicolas Williams wrote: > >One thing I find odd about IPS is that it has its own indexing scheme, > >and it doesn't strike me as a very good idea to home-grow your own when > >you can use any number of off-the-shelf ones... > > If you read the mailing list archives, you'll see that many > off-the-shelf solutions were considered. Unfortunately, they failed to > meet at least one of the following requirements: > > * low/controllable resource usage > > * minimal or no external dependencies > > * useable by both client and server > > * not detrimental to livecd
SQLite3 meets all of those requirements; it's tiny, has resource controls, no external dependencies outside libc, no reason why it couldn't be used by the client and server (though it's not a networked DB) and already on the livecd (because Firefox needs it). It's blindingly fast for things like "pkg search -l ls", as in Mike's example. > I won't claim there is an off-the-shelf solution that couldn't have been > used, but all of the other ones checked were problematic in one way or > another. For example: pyLucene requires a Java Runtime, MySQL would be > way over the top in dependencies and was not suitable for livecd, sqlite > doesn't really have the indexing/search capabilities desired, etc. What "indexing/search capabilities" does SQLite3 lack that you need? Sure, SQLite3 doesn't have hash-style indexes, so if you need to index fields with low cardinality (e.g., file permissions) it doesn't help, but I suspect that that's not the problem here (one doesn't search only by file permissions, or even by file permissions at all). It does have B-Tree-like indexes, pluggable collation functions, the ability to have multiple indexes per-table (it only uses one index per-table when running any one query though), and so on. We used SQLite3 extensively in the roboporter project. For the sorts of queries that IPS needs to do it's plenty fast enough, faster than IPS. > You'd have to ask Brock for the details, but I suspect he's already > posted about it before on this list. Brock? Nico -- _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
