> On Jul 7, 2016, at 7:06 PM, Jeff Hammond <jeff.scie...@gmail.com> wrote: > > > > On Thu, Jul 7, 2016 at 4:34 PM, Richard Mills <richardtmi...@gmail.com> wrote: > On Fri, Jul 1, 2016 at 4:13 PM, Jeff Hammond <jeff.scie...@gmail.com> wrote: > [...] > > Maybe I am just biased because I spend all of my time reading > www.nextplatform.com, but I hear machine learning is becoming an important > HPC workload. While the most hyped efforts related to running inaccurate - > the technical term is half-precision - dense matrix multiplication as fast as > possible, I suspect that more elegant approaches will prevail. Presumably > there is something that PETSc can do to enable machine learning algorithms. > As most of the existing approaches use silly programming models based on > MapReduce, it can't be too hard for PETSc to do better. > > "Machine learning" is definitely the hype du jour, but when that term gets > thrown around, everyone is equating it with neural networks with a lot of > layers ("deep learning"). That's why everyone is going on about half > precision dense matrix multiplication, as low accuracy works fine for some of > this stuff. The thing is, there are a a ton of machine-learning approaches > out there that are NOT neural networks, and I worry that everyone is too > ready to jump into specialized hardware for neural nets when maybe there are > better approaches out there. Regarding machine learning approaches that use > sparse matrix methods, I think that PETSc (plus SLEPc) provide pretty good > building blocks right now for these, though there are probably things that > could be better supported. But what machine learning approaches PETSc should > target right now, I don't know. Program managers currently like terms like > "neuromorphic computing"
It may be as much or even more idiots who talk to program managers that like "neuromorphic computing". > and half-precision computations seem to be the focus. (Though why stop > there? Why not quarter precision?!!) > > > Google TPU does quarter precision i.e. 8-bit fixed-point > [http://www.nextplatform.com/2016/05/19/google-takes-unconventional-route-homegrown-machine-learning-chips/], > so the machine learning folks have already gone there. No need to speculate > about it :-) > > Jeff > > -- > Jeff Hammond > jeff.scie...@gmail.com > http://jeffhammond.github.io/