On Thu, Jul 7, 2016 at 4:34 PM, Richard Mills <richardtmi...@gmail.com> wrote:
> On Fri, Jul 1, 2016 at 4:13 PM, Jeff Hammond <jeff.scie...@gmail.com> > wrote: > >> [...] >> >> Maybe I am just biased because I spend all of my time reading >> www.nextplatform.com, but I hear machine learning is becoming an >> important HPC workload. While the most hyped efforts related to running >> inaccurate - the technical term is half-precision - dense matrix >> multiplication as fast as possible, I suspect that more elegant approaches >> will prevail. Presumably there is something that PETSc can do to enable >> machine learning algorithms. As most of the existing approaches use silly >> programming models based on MapReduce, it can't be too hard for PETSc to do >> better. >> > > "Machine learning" is definitely the hype du jour, but when that term gets > thrown around, everyone is equating it with neural networks with a lot of > layers ("deep learning"). That's why everyone is going on about half > precision dense matrix multiplication, as low accuracy works fine for some > of this stuff. The thing is, there are a a ton of machine-learning > approaches out there that are NOT neural networks, and I worry that > everyone is too ready to jump into specialized hardware for neural nets > when maybe there are better approaches out there. Regarding machine > learning approaches that use sparse matrix methods, I think that PETSc > (plus SLEPc) provide pretty good building blocks right now for these, > though there are probably things that could be better supported. But what > machine learning approaches PETSc should target right now, I don't know. > Program managers currently like terms like "neuromorphic computing" and > half-precision computations seem to be the focus. (Though why stop there? > Why not quarter precision?!!) > > Google TPU does quarter precision i.e. 8-bit fixed-point [ http://www.nextplatform.com/2016/05/19/google-takes-unconventional-route-homegrown-machine-learning-chips/], so the machine learning folks have already gone there. No need to speculate about it :-) Jeff -- Jeff Hammond jeff.scie...@gmail.com http://jeffhammond.github.io/