Tim, On the surface you're right; but ultimately you're wrong, at least in my experience
Consider for instance the "machine learning for genomics" work I've been doing with some colleagues... Sure, we spend nearly all our budget on people not machines... However, it does often take several days for us to do a MOSES (machine learning algorithm) run across a body of datasets that interests us (using crossvalidation etc. etc.) .... Speeding that up by 10% wouldn't make much difference to us. Speeding that up by 100x would make a tremendous difference to us, because it would then let us perform MOSES-parameter-space search in a more automated way, rather than "largely by hand.".... Speeding that up by 10x would make a moderate difference to us.... Right now we could speed up MOSES runs by 10x or 100x by using, say, 15x or 150x more machines in the cloud (MOSES in principle runs distributed, with some performance penalty). This would require some legwork in dusting off the distributed-MOSES code, but not more than a months' work for that I guess. But that's not economically sensible at the moment, in terms of our project goals... But once hardware gets cheaper, so that using 10x as much computational firepower as we have now doesn't alter our budget substantially, then this WILL enable us to proceed scientifically much faster in certain directions. It will allow us to understand how MOSES depends on its parameters much better, which will allow us to improve MOSES a lot, or else maybe to design better algorithms... This is not an abstract theory point, this is a point borne of decades of practice, and I think it's similar to Matt Mahoney's point which is also borne of decades of practice on his part... -- Ben On Sat, Nov 29, 2014 at 11:05 AM, Tim Tyler via AGI <[email protected]> wrote: > On 27/11/2014 23:17, Matt Mahoney via AGI wrote: > >> I am intimately familiar with the process of developing data >> compression software. It is an iterative process. You think you have a >> good idea of what changes ought to improve compression. But then you >> do the experiment and you are right maybe less than half of the time. >> Even if you are a fast coder, you can see that development time is >> limited by the CPU power available to do the tests and gain a couple >> bits of knowledge. > > > It sounds unlikely to me. Most software companies spend an order of > magnitude > more on human resources than they do on computational capacity. In the case > of > compression, there are many possible changes which could be tested in > parallel - > and then the useful ones combined to form the basis of the next generation > of > the product. Even if the process has an exceptionally slow build-test cycle, > that > doesn't stop the project from developing in parallel - and absorbing human > resources in the form of programmers, testers, trainers, management, sales > and marketing. Indeed, the technical side of general-purpose data > compression > development is fairly easy to parallelise - due to all the different sorts > of data > that need to be compressed. Computers are pretty cheap these days. I think > that it is rare for the machines to be a serious bottleneck. > -- > __________ > |im |yler http://timtyler.org/ [email protected] Remove lock to reply. > > > > ------------------------------------------- > AGI > Archives: https://www.listbox.com/member/archive/303/=now > RSS Feed: https://www.listbox.com/member/archive/rss/303/212726-deec6279 > Modify Your Subscription: > https://www.listbox.com/member/?&; > Powered by Listbox: http://www.listbox.com -- Ben Goertzel, PhD http://goertzel.org "The reasonable man adapts himself to the world: the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man." -- George Bernard Shaw ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
