Xiao, One more timing result: The compute pattern optimized R version takes about 30 sec meaning it is 3-4X slower than J and about 50% slower than non compute pattern optimized C and 10X slower than compute pattern optimized C.
On Tue, Apr 18, 2017 at 8:21 PM, Michael Goodrich < [email protected]> wrote: > Xiao, > > 200 training patterns, 4 inputs, 1 output each, 3 hidden neurodes: > > (SWI Prolog / Bash) > > swipl -s bamc.4c.pl -g main --f 'f1.csv' --n 9999 --o c --h 3 > > or > > F0=$(mktemp)`; F=$(mktemp); runswipl bamc.4c.pl main 9999 $F0 200 > 'f1.csv' 3 2&> $F > > > runtimes: > C ~4 sec > J ~ 9sec > Prolog ~ 3156 sec > > J: > 0!:0 <'tlp0.j' > 6!:2 'f tlpmcmce10 1e4' > > > > On Tue, Apr 18, 2017 at 1:19 PM, Xiao-Yong Jin <[email protected]> > wrote: > >> > On Apr 18, 2017, at 12:08 PM, Michael Goodrich < >> [email protected]> wrote: >> > >> > Actually I don't mind sharing the current J version. It is immature >> and little more than a transliteration of the basic C code. I hope >> eventually to have it all loopless, etc with proper ranking and tacitness. >> I have both Prolog and Haskell versions but J is somewhat different than >> Haskell - in all the right ways. >> >> It would be interesting to see your prolog version. How is the >> performance there? >> >> > On Apr 18, 2017, at 12:09 PM, Michael Goodrich < >> [email protected]> wrote: >> > >> > Btw please note that the defined monad 'sgmd' is *outperforming* the >> primitive tanh ! >> >> Interesting. 7&o. always performs better in my test. How do you >> actually use it? >> >> > >> > Sent from my iPhone >> > >> >> On Apr 18, 2017, at 11:31 AM, Xiao-Yong Jin <[email protected]> >> wrote: >> >> >> >> >> >>> On Apr 18, 2017, at 9:23 AM, Michael Goodrich < >> [email protected]> wrote: >> >>> >> >>> Hi Henry, >> >>> >> >>> Thanks for your interest. I owe you some better information. >> >>> >> >>> First off its not really an apples-apples comparison as the C version >> is >> >>> very mature with some performance tricks designed to reduce >> calculations to >> >>> the bare minimum (e.g., do not recalculate matrices but instead do >> selected >> >>> in place updates as necessary). This gave me an 5-6X speed >> improvement in >> >>> the C version. >> >> >> >> If you are only optimizing away unnecessary computations in C, your >> code will not >> >> really beat a well written J code. There are still a lot more you can >> do in C. >> >> >> >>> When I attempted to put same in the J code it ran SLOWER >> >>> than simply recalculating entire matrix although in many cases only a >> >>> column was actually updated, so i backed them out. >> >> >> >> J does reasonable inplace (avoiding allocating and copying) updates. >> >> Look them up and see if you can better employ those. In an interpreted >> >> mostly functional language like J, you want to minimize reallocating >> arrays >> >> and moving whole array. Too much copying hurts more than extra >> floating >> >> points operations. >> >> >> >>> This is a Markov Chain Monte Carlo Bayesian Artificial Neural Network >> >>> (Three Layer Perceptron) application that in the test problem produces >> >>> about 1e5 chain states (not saving them but streaming them to another >> C >> >>> prog) using a half dozen matrices the largest of which (for this test >> >>> problem) is about 200x5 >> >> >> >> It's really small and fits in L1 cache. Naive C loops would have no >> problem. >> >> If you go larger than L2 cache, you will need to call dgemm or any >> cache >> >> friendly block multiplication algorithm for better performance. >> >> >> >>> >> >>> Another curiosity is that in the C version using a (user defined) >> sigmoid >> >>> vice 'tanh' as the non linear activation (on all matrix elements) >> expands >> >>> the run time by 1.75X. In the J version the same choice *reduces* >> run time >> >>> by about 20% over the 'tanh' J primitive (?). >> >>> >> >>> sgmd =. monad : '1%(1+^-y)' >> >> >> >> In any interpreted language, primitives are always going to outperform >> composite >> >> functions. >> >> >> >>> As far as releasing code, this is the outgrowth of my dissertation >> work and >> >>> I may hope someday to commercialize and so I reluctant to release >> it. Pity >> >>> - I know you cant be sure what I doing to in order to diagnose the >> >>> situation, but perhaps we can find a way to accomplish what you want >> any >> >>> way by pursuing this together. >> >> >> >> I don't see a good MCMC code in J, so I'm also developing my own. >> Perhaps we >> >> cal share that part at some point so you don't have to give away your >> baby neural network. >> >> >> >> ---------------------------------------------------------------------- >> >> For information about J forums see http://www.jsoftware.com/forums.htm >> > ---------------------------------------------------------------------- >> > For information about J forums see http://www.jsoftware.com/forums.htm >> >> ---------------------------------------------------------------------- >> For information about J forums see http://www.jsoftware.com/forums.htm >> > > > > -- > Dominus enim Iesus Ecclesiae Suae et, > > -Michael > > > "Let your religion be less of a theory and more of a love affair." > - G.K. Chesterton > -- Dominus enim Iesus Ecclesiae Suae et, -Michael "Let your religion be less of a theory and more of a love affair." - G.K. Chesterton ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
