The best way to handle situations like this is to under-promise and over-deliver. Keep expectations low. If we are correct, then over time our work will be recognized and adopted. We dont have to hype things or make a lot of claims.
The be patient attitude doesnt always work. For example in the race between MySpace and Facebook there was not a right way to build a huge social networking business. Someone had to win and it could have gone either way. Whoever pushed hardest, adapted faster, and marketed better, won. Machine intelligence isnt like this. In the end there arent multiple ways of going about it. Looking back 20 or 50 years from now there wont be multiple ways of building intelligent machines. It wont be it could have gone this way or it could have gone that way. Intelligent machines will be based on on-line learning, SDRs, hierarchies, and distributed temporal memory will be a big part of it. I am so confident in this approach that I feel we can be patient. Time is on our side. Of course we need to apply the CLA, test it, show what it can do and what it cant do. But I dont worry about what other people claim or the noise they generate. Jeff From: nupic [mailto:[email protected]] On Behalf Of Ian Danforth Sent: Tuesday, October 29, 2013 2:32 AM To: NuPIC general mailing list. Subject: Re: [nupic-dev] Vicarious AI breaks CAPTCHA Turing test Pedro, Thanks for the link there is a lot of honest, direct and useful feedback in that thread from Dr. LeCun and others. This is a cautionary tale for our community. Mainline AI/ML research has been maligned so often it now takes extraordinary results to make even modest claims. To garner greater acceptance for NuPIC and CLA we need replicable results at or above anything out there on accepted problems and datasets. Luckily we're open source so no one can claim we're hiding anything. In some ways this is unfortunate as many of CLAs unique strengths (inherently online learning, variable and high order sequence memory, temporal pooling) could be missed or ignored by the wider research field until we tackle some of the classic problems head on. Ian On Oct 28, 2013 12:12 PM, "Pedro Tabacof" <[email protected]> wrote: Yann LeCun (deep learning expert) offers a skeptical view: AI startup Vicarious claims to have a system that can solve CAPTCHAs with "succes rate up to 90%". Beware: It's a textbook example of AI hype of the worst kind Hype is dangerous to AI. Hype killed AI four times in the last five decades. AI Hype must be stopped. Perhaps Vicarious can get "up to 90%" accuracy on some CAPTCHA dataset they cooked up, but - (1) breaking CAPTCHAs is hardly an interesting task, unless you are a spammer; - (2) it's easy to claim success on a dataset you cooked up yourself. There is no risk someone else will beat you. - (3) recognizing object in images is much, much harder than breaking CAPTCHAs. Some deep learning systems can already do this with decent accuracy. Some such systems have been deployed by Google and Baidu. - (4) doing simultaneous segmentation and recognition of character strings is hardly a breakthrough. See demos of a 20 year-old system here: http://yann.lecun.com/exdb/lenet/index.html The sad thing is that this announcement is being picked up by a number publications, including MIT Tech Review, Forbes, etc. Here is an advice to scientific/tech journalists: please, please do not believe vague claims by AI startupsunless they produce state of the art results on widely accepted benchmarks. This is particularly true for claims in image and speech recognition for which good benchmarks exists. For image recognition, a good example of such benchmark would be the ImageNet Large Scale Visual Recognition Challenge. Whenever a startup claims "90% accuracy" on some random task, do not consider this newsworthy. If the company also makes claims like "we are developing machine learning software based on the computational principles of the human brain" or uses impressive-sounding names like "Recursive Cortical Network", be even more suspicious. There are extremely impressive applications of deep learning out there (e.g. deployed by Google, Baidu, Microsoft, IBM, and a few startups), but this is not one of them. Google's automatic photo tagger and Baidu's image retrieval system are much, much more impressive than the system in this announcement. Even if we just talk about challenging character recognition tasks, Google's system for picking out house numbers in StreetView images is way more impressive than this. AI "died" about four times in five decades because of hype: people made wild claims (often to impress potential investors or funding agencies) and could not deliver. Backlash ensued. It happened twice with neural nets already: once in the late 60's and again in the mid-90's. Don't let it happen again. Beware of hype. And by the way, no one is interested in breaking CAPTCHAs except spammers and computer security researchers. That's why you won't find many computer vision papers on the topic. That's also why it would be easy to break records, even if a standard dataset existed. Dileep George's rebuttal: Hi Yann, (1) CAPTCHA contains many of the problems that make general vision hardhttp://tinyurl.com/mkhllyu. We will be publishing results on standard benchmarks in the future as well. (2) We get 90% pass rate on a validation set of 10,000 captchas downloaded from reCAPTCHA on Nov 5 at 11:25AM. You can download the data for yourself here:https://www.dropbox.com/s/sqr7b6ck0bzt0ur/recaptcha10k.zip (3) We recognize objects in images too, this is just one demo of our system. (4) Looks like you linked to the wrong video, because the letters in that video look pretty well separated and easily segmented out. I'd like to see any current system parse modern CAPTCHAs. Out of curiosity, did you also have this reaction to the news about Watson? It's good to (sometimes) post results that the average person can connect with. Source: https://plus.google.com/104362980539466846301/posts/Qwj9EEkUJXY On Mon, Oct 28, 2013 at 5:03 PM, Azat <[email protected]> wrote: http://www.kurzweilai.net/vicarious-ai-breaks-captcha-turing-test Azat _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org -- Pedro Tabacof, Unicamp - Eng. de Computação 08. _______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________ nupic mailing list [email protected] http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
