I've just skimmed it and so probably missed some key details, but this looks like a hidden layer model where you just randomly pick values for the hidden layer parameters, and then solve a simple linear regression model to predict outputs from the randomized hidden layer. The random values are never tuned or learned. It sounds too good to be true at first, and the test results show it does worse on regression tasks (?) but it gets close and is simple.
Maybe you could think of it as an ensemble type of approach. You make a bunch of random projections of the input, each of which is then used to solve a different regression problem for the same output. Those answers are combined via weighted that you learn with one step. On Tue, Apr 30, 2013 at 2:20 PM, Reto Matter <[email protected]> wrote: > As far as I understand ELMs, the main difference is that learning in that > particular setting comes down to 3 relatively simple steps and in fact no > iteration as in other learning algos (e.g. Backpropagation) is needed. So, > in that respect, the learning phase is blazingly fast compared to other > approaches. > I don't think they are any better in terms of generalization capabilities, > but I haven't studied the theory behind ELMs good enough to really be > sure... > > greets, > reto > > > On Tue, Apr 30, 2013 at 2:45 PM, Louis Hénault <[email protected]>wrote: > >> I am not at home where I have my courses note about it, but you can have a >> look here for example: >> http://msrvideo.vo.msecnd.net/rmcvideos/144113/dl/144113.pdf >> page 50 you have a comparison between SVM and ELM, and ELM outperform SVM >> for the testing and training times. >> >> It is not easy to give theoretical reasons why ELM are so quick compared to >> SVM, but they are. >> >> If someone seems to be interested to work on it with me, just tell me. >> >> >> >> 2013/4/30 Sean Owen <[email protected]> >> >> > If you care to work on it, you should work on it. Implementations >> > exist or don't exist because someone created it, or nobody was >> > interested in creating it. >> > >> > I have never heard of 'extreme learning' and found this summary: >> > >> > >> http://www.slideshare.net/formatc666/extreme-learning-machinetheory-and-applications >> > >> > If it's accurate, this is just describing a single hidden layer model >> > trained with back propagation. I don't see what's new? the part about >> > learning the beta weights is simple linear algebra. >> > >> > If it's just a hidden layer model, it's not necessarily better than SVMs, >> > no. >> > >> > On Tue, Apr 30, 2013 at 11:05 AM, Louis Hénault <[email protected] >> > >> > wrote: >> > > Hi everybody, >> > > >> > > Many people are trying to integrate SVM to Mahout. I can understand >> since >> > > SVM are really efficient in a "small data" context. >> > > But, as you may know, SVM has: >> > > -a slow learning speed >> > > -a poor learning scalability >> > > >> > > In contrast, ELM give results which are usually at least as good as >> SVM's >> > > and are something like 1000x faster. >> > > So, why not trying to work on this topic? >> > > >> > > (Sorry if someone already talked about it, I'm new on this mailing and >> > did >> > > not find anything after some researches) >> > > >> > > Regards >> > >>
