On Thu, Jul 7, 2016 at 5:06 PM, Jeff Hammond <jeff.scie...@gmail.com> wrote:
> > > On Thu, Jul 7, 2016 at 4:34 PM, Richard Mills <richardtmi...@gmail.com> > wrote: > >> On Fri, Jul 1, 2016 at 4:13 PM, Jeff Hammond <jeff.scie...@gmail.com> >> wrote: >> >>> [...] >>> >>> Maybe I am just biased because I spend all of my time reading >>> www.nextplatform.com, but I hear machine learning is becoming an >>> important HPC workload. While the most hyped efforts related to running >>> inaccurate - the technical term is half-precision - dense matrix >>> multiplication as fast as possible, I suspect that more elegant approaches >>> will prevail. Presumably there is something that PETSc can do to enable >>> machine learning algorithms. As most of the existing approaches use silly >>> programming models based on MapReduce, it can't be too hard for PETSc to do >>> better. >>> >> >> "Machine learning" is definitely the hype du jour, but when that term >> gets thrown around, everyone is equating it with neural networks with a lot >> of layers ("deep learning"). That's why everyone is going on about half >> precision dense matrix multiplication, as low accuracy works fine for some >> of this stuff. The thing is, there are a a ton of machine-learning >> approaches out there that are NOT neural networks, and I worry that >> everyone is too ready to jump into specialized hardware for neural nets >> when maybe there are better approaches out there. Regarding machine >> learning approaches that use sparse matrix methods, I think that PETSc >> (plus SLEPc) provide pretty good building blocks right now for these, >> though there are probably things that could be better supported. But what >> machine learning approaches PETSc should target right now, I don't know. >> Program managers currently like terms like "neuromorphic computing" and >> half-precision computations seem to be the focus. (Though why stop there? >> Why not quarter precision?!!) >> >> > Google TPU does quarter precision i.e. 8-bit fixed-point [ > http://www.nextplatform.com/2016/05/19/google-takes-unconventional-route-homegrown-machine-learning-chips/], > so the machine learning folks have already gone there. No need to > speculate about it :-) > How wonderfully retro! I remember doing stuff like this for 3D graphics, back in the day when floating point was way too expensive, so we had to do it with fixed point calculations. I guess I'm getting pretty old in computing years... --Richard > > Jeff > > -- > Jeff Hammond > jeff.scie...@gmail.com > http://jeffhammond.github.io/ >