> On Jul 7, 2016, at 7:06 PM, Jeff Hammond <jeff.scie...@gmail.com> wrote:
>
>
>
> On Thu, Jul 7, 2016 at 4:34 PM, Richard Mills <richardtmi...@gmail.com> wrote:
> On Fri, Jul 1, 2016 at 4:13 PM, Jeff Hammond <jeff.scie...@gmail.com> wrote:
> [...]
>
> Maybe I am just biased because I spend all of my time reading
> www.nextplatform.com, but I hear machine learning is becoming an important
> HPC workload. While the most hyped efforts related to running inaccurate -
> the technical term is half-precision - dense matrix multiplication as fast as
> possible, I suspect that more elegant approaches will prevail. Presumably
> there is something that PETSc can do to enable machine learning algorithms.
> As most of the existing approaches use silly programming models based on
> MapReduce, it can't be too hard for PETSc to do better.
>
> "Machine learning" is definitely the hype du jour, but when that term gets
> thrown around, everyone is equating it with neural networks with a lot of
> layers ("deep learning"). That's why everyone is going on about half
> precision dense matrix multiplication, as low accuracy works fine for some of
> this stuff. The thing is, there are a a ton of machine-learning approaches
> out there that are NOT neural networks, and I worry that everyone is too
> ready to jump into specialized hardware for neural nets when maybe there are
> better approaches out there. Regarding machine learning approaches that use
> sparse matrix methods, I think that PETSc (plus SLEPc) provide pretty good
> building blocks right now for these, though there are probably things that
> could be better supported. But what machine learning approaches PETSc should
> target right now, I don't know. Program managers currently like terms like
> "neuromorphic computing"
It may be as much or even more idiots who talk to program managers that like
"neuromorphic computing".
> and half-precision computations seem to be the focus. (Though why stop
> there? Why not quarter precision?!!)
>
>
> Google TPU does quarter precision i.e. 8-bit fixed-point
> [http://www.nextplatform.com/2016/05/19/google-takes-unconventional-route-homegrown-machine-learning-chips/],
> so the machine learning folks have already gone there. No need to speculate
> about it :-)
>
> Jeff
>
> --
> Jeff Hammond
> jeff.scie...@gmail.com
> http://jeffhammond.github.io/
