--- Ben Peterson <[EMAIL PROTECTED]> wrote: > Maybe I'm hallucinating, but I thought I read somewhere of some test > stronger or more reliable than the Turing Test to verify whether or > not a machine had achieved human-level intelligence.
Text compression? http://cs.fit.edu/~mmahoney/compression/rationale.html I wouldn't say it is more powerful, just more objective and repeatable. Also, in its present form it can only be used to compare one model to another. To test whether a model achieves "human" level, it needs to be compared to average human ability to predict successive words or symbols in a text stream. This is a harder test to get right, one I have not yet attempted. Shannon [1] first did this test in 1950 but left a wide range of uncertainty (0.6 to 1.3 bits per character) due to his method of converting a ranking of next-letter guesses to a probability distribution. Cover and King [2] reduced the uncertainty in 1978 (upper bound of 1.3 bpc) by making the probability distribution explicit in a gambling game, but their method is time consuming and could only be used on a small sample of text. I have also made some attempts to refine Shannon's method in http://cs.fit.edu/~mmahoney/dissertation/entropy1.html (under 1.1 bpc). In any case, none of these measurements were on the actual test data used in my large text benchmark. The best result to date is 1.04 bpc, but I would not call this AI. I know these programs use rather simple language models and are memory bound. (The top program needs 4.6 GB). The Wikipedia data set I use probably has a lower entropy than the data used in the literature, possibly 0.8-0.9 bpc. That's just a guess, because as I said, I don't yet have a reliable way to measure it. References 1. Shannon, Cluade E., Prediction and Entropy of Printed English, Bell Sys. Tech. J (3) p. 50-64, 1950. 2. Cover, T. M., and R. C. King, A Convergent Gambling Estimate of the Entropy of English, IEEE Transactions on Information Theory (24)4 (July) pp. 413-421, 1978. -- Matt Mahoney, [EMAIL PROTECTED] ------------------------------------------- singularity Archives: http://www.listbox.com/member/archive/11983/=now RSS Feed: http://www.listbox.com/member/archive/rss/11983/ Modify Your Subscription: http://www.listbox.com/member/?member_id=4007604&id_secret=98631122-712fa4 Powered by Listbox: http://www.listbox.com
