--- "YKY (Yan King Yin)" <[EMAIL PROTECTED]> wrote: > For those using database systems for AGI, I'm wondering if the data > retrieval rate would be a problem.
I analyzed the scalability of distributed indexing for my thesis (linked at http://www.mattmahoney.net/agi.html ). For data randomly distributed in a vector space model up to log n dimensions, storage is O(n log n), retrieval time is effectively O(log n) and update is O(log^2 n). In practice you can do better because data tends to cluster, reducing the effective number of dimensions, and because accesses tend to be distributed nonuniformly. Data accessed frequently will tend to be cached in nearby nodes. I realize you are asking about the relational model, but you can implement the most common transactions, e.g. retrieving or updating a small number of records at a time, by storing records of the form "author timestamp table field=value field=value ...". This also gives you transaction logging, rollback, and authentication, which will be important in any database with lots of users (I assume AGI). However I don't think it will be as powerful as records of the form "author timestamp arbitrary_text". > To use an example, > > If a lot of people search for "Harry Porter", then a conventional > database system would make future retrieval of the "Harry Porter" node > faster. > > But the requirement of the inference system is such that, if "Harry > Porter" is fetched, then we would want *other* things that are > associated with Harry Porter to be retrieved faster in the future, for > example items such as "JK Rowling" or "fantasy fiction". A huge relational database would retrieve the fact that Harry Porter won a gold medal for the high jump in the 1908 Olympics. A better language model (like Google) might figure out that you meant "Harry Potter" :-) -- Matt Mahoney, [EMAIL PROTECTED] ------------------------------------------- agi Archives: http://www.listbox.com/member/archive/303/=now RSS Feed: http://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: http://www.listbox.com/member/?member_id=8660244&id_secret=101455710-f059c4 Powered by Listbox: http://www.listbox.com
