I agree with Jeff on the under-promise, over-deliver plan.
What we're trying to do here is gradually build out the functionality in the
same way the neocortex evolved (hopefully a little faster). The CLA is a
general-purpose piece of cortex-like technology, which is only going to be
reasonably good at a few things at first, and not very good at the marquee
tasks.
I think Geoff Hinton is showing the way here. His model is much less
neuroscientific (but there is some correspondence), but it allows him to build
big, deep hierarchies which demonstrate real power. Models based on his work
(by him, his students and colleagues, including Jann) have won several very big
competitions. Chastened somewhat after the connectionist bubble if the early
90's, he's quite modest despite the impressive progress he's demonstrating.
Both Jeff and Geoff are only too happy to show everyone how their stuff works,
and want us all to build on their work to see how much can be achieved. It's
kind of the entire point in both cases.
As mentioned by several people, the history of AI is littered with the debris
of broken promises. While I wish these guys luck, I don't see the kind of body
language which would hope for.
The CAPTCHA example is unlikely to be the result of just plugging together a
few general-purpose nets. As Jeff says, it is more likely based on a lot of
tuning and design work, and every other application will likely take a huge
amount of work to get performance.
Geoff (and his coworkers), on the other hand, just put together simple
components and do some nannying to get them to achieve quite similar prowess in
multiple areas. Then they publish a paper explains exactly how they did it.
My belief is that the brain uses Jeff's CLA (or a close relation) at the region
level, and the hierarchy and large-scale structure resembles Geoff's models
somewhat.
The tortoise is very, very likely to win the race.
Regards,
Fergal Byrne
—
Sent from Mailbox for iPhone
On Tue, Oct 29, 2013 at 5:56 PM, Jeff Hawkins <[email protected]>
wrote:
> The best way to handle situations like this is to under-promise and
> over-deliver. Keep expectations low. If we are correct, then over time our
> work will be recognized and adopted. We dont have to hype things or make a
> lot of claims.
>
> The be patient attitude doesnt always work. For example in the race
> between MySpace and Facebook there was not a right way to build a huge
> social networking business. Someone had to win and it could have gone
> either way. Whoever pushed hardest, adapted faster, and marketed better,
> won.
>
> Machine intelligence isnt like this. In the end there arent multiple ways
> of going about it. Looking back 20 or 50 years from now there wont be
> multiple ways of building intelligent machines. It wont be it could have
> gone this way or it could have gone that way. Intelligent machines will
> be based on on-line learning, SDRs, hierarchies, and distributed temporal
> memory will be a big part of it. I am so confident in this approach that I
> feel we can be patient. Time is on our side.
>
> Of course we need to apply the CLA, test it, show what it can do and what it
> cant do. But I dont worry about what other people claim or the noise they
> generate.
> Jeff
>
> From: nupic [mailto:[email protected]] On Behalf Of Ian
> Danforth
> Sent: Tuesday, October 29, 2013 2:32 AM
> To: NuPIC general mailing list.
> Subject: Re: [nupic-dev] Vicarious AI breaks CAPTCHA Turing test
>
> Pedro,
> Thanks for the link there is a lot of honest, direct and useful feedback in
> that thread from Dr. LeCun and others.
> This is a cautionary tale for our community. Mainline AI/ML research has
> been maligned so often it now takes extraordinary results to make even
> modest claims.
> To garner greater acceptance for NuPIC and CLA we need replicable results at
> or above anything out there on accepted problems and datasets. Luckily we're
> open source so no one can claim we're hiding anything.
> In some ways this is unfortunate as many of CLAs unique strengths
> (inherently online learning, variable and high order sequence memory,
> temporal pooling) could be missed or ignored by the wider research field
> until we tackle some of the classic problems head on.
> Ian
> On Oct 28, 2013 12:12 PM, "Pedro Tabacof" <[email protected]> wrote:
> Yann LeCun (deep learning expert) offers a skeptical view:
>
> AI startup Vicarious claims to have a system that can solve CAPTCHAs with
> "succes rate up to 90%".
> Beware: It's a textbook example of AI hype of the worst kind
> Hype is dangerous to AI. Hype killed AI four times in the last five decades.
> AI Hype must be stopped.
> Perhaps Vicarious can get "up to 90%" accuracy on some CAPTCHA dataset they
> cooked up, but
> - (1) breaking CAPTCHAs is hardly an interesting task, unless you are a
> spammer;
> - (2) it's easy to claim success on a dataset you cooked up yourself. There
> is no risk someone else will beat you.
> - (3) recognizing object in images is much, much harder than breaking
> CAPTCHAs. Some deep learning systems can already do this with decent
> accuracy. Some such systems have been deployed by Google and Baidu.
> - (4) doing simultaneous segmentation and recognition of character strings
> is hardly a breakthrough. See demos of a 20 year-old system here:
> http://yann.lecun.com/exdb/lenet/index.html
> The sad thing is that this announcement is being picked up by a number
> publications, including MIT Tech Review, Forbes, etc.
> Here is an advice to scientific/tech journalists: please, please do not
> believe vague claims by AI startupsunless they produce state of the art
> results on widely accepted benchmarks.
> This is particularly true for claims in image and speech recognition for
> which good benchmarks exists. For image recognition, a good example of such
> benchmark would be the ImageNet Large Scale Visual Recognition Challenge.
> Whenever a startup claims "90% accuracy" on some random task, do not
> consider this newsworthy. If the company also makes claims like "we are
> developing machine learning software based on the computational principles
> of the human brain" or uses impressive-sounding names like "Recursive
> Cortical Network", be even more suspicious.
> There are extremely impressive applications of deep learning out there (e.g.
> deployed by Google, Baidu, Microsoft, IBM, and a few startups), but this is
> not one of them.
> Google's automatic photo tagger and Baidu's image retrieval system are much,
> much more impressive than the system in this announcement. Even if we just
> talk about challenging character recognition tasks, Google's system for
> picking out house numbers in StreetView images is way more impressive than
> this.
> AI "died" about four times in five decades because of hype: people made wild
> claims (often to impress potential investors or funding agencies) and could
> not deliver. Backlash ensued. It happened twice with neural nets already:
> once in the late 60's and again in the mid-90's.
> Don't let it happen again. Beware of hype.
> And by the way, no one is interested in breaking CAPTCHAs except spammers
> and computer security researchers. That's why you won't find many computer
> vision papers on the topic. That's also why it would be easy to break
> records, even if a standard dataset existed.
>
> Dileep George's rebuttal:
>
> Hi Yann,
> (1) CAPTCHA contains many of the problems that make general vision
> hardhttp://tinyurl.com/mkhllyu. We will be publishing results on standard
> benchmarks in the future as well.
> (2) We get 90% pass rate on a validation set of 10,000 captchas downloaded
> from reCAPTCHA on Nov 5 at 11:25AM. You can download the data for yourself
> here:https://www.dropbox.com/s/sqr7b6ck0bzt0ur/recaptcha10k.zip
> (3) We recognize objects in images too, this is just one demo of our system.
> (4) Looks like you linked to the wrong video, because the letters in that
> video look pretty well separated and easily segmented out. I'd like to see
> any current system parse modern CAPTCHAs.
> Out of curiosity, did you also have this reaction to the news about Watson?
> It's good to (sometimes) post results that the average person can connect
> with.
>
>
> Source: https://plus.google.com/104362980539466846301/posts/Qwj9EEkUJXY
>
>
> On Mon, Oct 28, 2013 at 5:03 PM, Azat <[email protected]> wrote:
> http://www.kurzweilai.net/vicarious-ai-breaks-captcha-turing-test
> Azat
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
>
> --
> Pedro Tabacof,
> Unicamp - Eng. de Computação 08.
> _______________________________________________
> nupic mailing list
> [email protected]
> http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
_______________________________________________
nupic mailing list
[email protected]
http://lists.numenta.org/mailman/listinfo/nupic_lists.numenta.org
