On 11 Sep, 19:31, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]> wrote: > > An acquaintance suggests that defragmentation would be a useful > service to provide along with memory management too, which also > requires an index.
I presume that you mean efficient access to large amounts of data in the sense that if all the data you want happens to be in the same page or segment, then retrieving it is much more efficient than having to seek around for all the different pieces. So the defragmentation would be what they call clustering in a relational database context: http://www.postgresql.org/docs/8.3/static/sql-cluster.html I've seen similar phenomena outside the relational database world, notably with big Lucene indexes which wouldn't fit in memory in their entirety. > I encourage overlap between a bare-bones alloc/free module and > established database territory and I'm very aware of it. > > Databases already support both concurrency and persistence, but don't > tell me you'd use a database for IPC. Of course, databases are widely used in scalable systems to hold central state, which is why there's a lot of effort put into to not only scaling up database installations, but also into things like caching which are supposed to save the database systems behind popular Web applications from excessive load. > And don't tell me you've never > wished you had a reference to a record in a table so that you could > make an update just by changing one word of memory at the right > place. Sometimes databases are overkill where all you want is dynamic > allocation. I think that the challenge is to reduce an abstract operation (for example, wanting to update a particular column in a particular record) to its measurable effects (this word of memory/disk will change as a consequence). It's easy for a human with a reasonable knowledge of, say, a relational database system to anticipate such things, but to actually collapse a number of layers through some kind of generic optimisation process is a lot more difficult. Paul -- http://mail.python.org/mailman/listinfo/python-list