Xiao, 200 training patterns, 4 inputs, 1 output each, 3 hidden neurodes:
(SWI Prolog / Bash) swipl -s bamc.4c.pl -g main --f 'f1.csv' --n 9999 --o c --h 3 or F0=$(mktemp)`; F=$(mktemp); runswipl bamc.4c.pl main 9999 $F0 200 'f1.csv' 3 2&> $F runtimes: C ~4 sec J ~ 9sec Prolog ~ 3156 sec J: 0!:0 <'tlp0.j' 6!:2 'f tlpmcmce10 1e4' On Tue, Apr 18, 2017 at 1:19 PM, Xiao-Yong Jin <[email protected]> wrote: > > On Apr 18, 2017, at 12:08 PM, Michael Goodrich < > [email protected]> wrote: > > > > Actually I don't mind sharing the current J version. It is immature and > little more than a transliteration of the basic C code. I hope eventually > to have it all loopless, etc with proper ranking and tacitness. I have > both Prolog and Haskell versions but J is somewhat different than Haskell - > in all the right ways. > > It would be interesting to see your prolog version. How is the > performance there? > > > On Apr 18, 2017, at 12:09 PM, Michael Goodrich < > [email protected]> wrote: > > > > Btw please note that the defined monad 'sgmd' is *outperforming* the > primitive tanh ! > > Interesting. 7&o. always performs better in my test. How do you actually > use it? > > > > > Sent from my iPhone > > > >> On Apr 18, 2017, at 11:31 AM, Xiao-Yong Jin <[email protected]> > wrote: > >> > >> > >>> On Apr 18, 2017, at 9:23 AM, Michael Goodrich < > [email protected]> wrote: > >>> > >>> Hi Henry, > >>> > >>> Thanks for your interest. I owe you some better information. > >>> > >>> First off its not really an apples-apples comparison as the C version > is > >>> very mature with some performance tricks designed to reduce > calculations to > >>> the bare minimum (e.g., do not recalculate matrices but instead do > selected > >>> in place updates as necessary). This gave me an 5-6X speed > improvement in > >>> the C version. > >> > >> If you are only optimizing away unnecessary computations in C, your > code will not > >> really beat a well written J code. There are still a lot more you can > do in C. > >> > >>> When I attempted to put same in the J code it ran SLOWER > >>> than simply recalculating entire matrix although in many cases only a > >>> column was actually updated, so i backed them out. > >> > >> J does reasonable inplace (avoiding allocating and copying) updates. > >> Look them up and see if you can better employ those. In an interpreted > >> mostly functional language like J, you want to minimize reallocating > arrays > >> and moving whole array. Too much copying hurts more than extra floating > >> points operations. > >> > >>> This is a Markov Chain Monte Carlo Bayesian Artificial Neural Network > >>> (Three Layer Perceptron) application that in the test problem produces > >>> about 1e5 chain states (not saving them but streaming them to another C > >>> prog) using a half dozen matrices the largest of which (for this test > >>> problem) is about 200x5 > >> > >> It's really small and fits in L1 cache. Naive C loops would have no > problem. > >> If you go larger than L2 cache, you will need to call dgemm or any cache > >> friendly block multiplication algorithm for better performance. > >> > >>> > >>> Another curiosity is that in the C version using a (user defined) > sigmoid > >>> vice 'tanh' as the non linear activation (on all matrix elements) > expands > >>> the run time by 1.75X. In the J version the same choice *reduces* run > time > >>> by about 20% over the 'tanh' J primitive (?). > >>> > >>> sgmd =. monad : '1%(1+^-y)' > >> > >> In any interpreted language, primitives are always going to outperform > composite > >> functions. > >> > >>> As far as releasing code, this is the outgrowth of my dissertation > work and > >>> I may hope someday to commercialize and so I reluctant to release it. > Pity > >>> - I know you cant be sure what I doing to in order to diagnose the > >>> situation, but perhaps we can find a way to accomplish what you want > any > >>> way by pursuing this together. > >> > >> I don't see a good MCMC code in J, so I'm also developing my own. > Perhaps we > >> cal share that part at some point so you don't have to give away your > baby neural network. > >> > >> ---------------------------------------------------------------------- > >> For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm > -- Dominus enim Iesus Ecclesiae Suae et, -Michael "Let your religion be less of a theory and more of a love affair." - G.K. Chesterton
1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4 1e_4
1e_4 1e_4 1e_4 1e_4
-100 -9.12945250727628 4.53999297624849E-05 10 -99 -8.13673737507105 0.5168513344917 9.801 -98 -6.81963620068136 0.52030852120631 9.604 -97 -5.23065765157699 0.52378883282973 9.409 -96 -3.43314928819899 0.52729242404305 9.216 -95 -1.49877209662952 0.53081945056201 9.025 -94 0.49535640878367 0.53437006914394 8.836 -93 2.46973661736621 0.53794443759467 8.649 -92 4.34565622071894 0.54154271477563 8.464 -91 6.04832822406284 0.54516506061084 8.281 -90 7.50987246771676 0.54881163609403 8.1 -89 8.67202179485581 0.55248260329583 7.921 -88 9.48844497918124 0.55617812537094 7.744 -87 9.92659380470633 0.5598983665654 7.569 -86 9.96900066041596 0.56364349222387 7.396 -85 9.61397491879557 0.567413668797 7.225 -84 8.87567033581505 0.57120906384881 7.056 -83 7.78352078534298 0.57502984606416 6.889 -82 6.3810668234795 0.5788761852562 6.724 -81 4.72421986398466 0.58274825237399 6.561 -80 2.87903316665065 0.58664621951003 6.4 -79 0.91906850227682 0.59057025990795 6.241 -78 -1.07753652299442 0.59452054797019 6.084 -77 -3.03118356745702 0.59849725926577 5.929 -76 -4.86398688853798 0.60250057053804 5.776 -75 -6.50287840157117 0.60653065971263 5.625 -74 -7.88252067375316 0.61058770590527 5.476 -73 -8.94791172140503 0.6146718894298 5.329 -72 -9.65657776549277 0.61878339180614 5.184 -71 -9.98026652716362 0.62292239576842 5.041 -70 -9.9060735569487 0.62708908527306 4.9 -69 -9.43695669444105 0.63128364550693 4.761 -68 -8.59161814856497 0.63550626289562 4.624 -67 -7.40375889952449 0.63975712511172 4.489 -66 -5.92073514707224 0.64403642108314 4.356 -65 -4.20167036826641 0.64834434100151 4.225 -64 -2.31509825101539 0.65268107633065 4.096 -63 -0.33623047221139 0.65704681981506 3.969 -62 1.65604175448309 0.6614417654885 3.844 -61 3.58229282236827 0.66586610868263 3.721 -60 5.36572918000435 0.67032004603564 3.6 -59 6.93525084777122 0.67480377550103 3.481 -58 8.22828594968708 0.67931749635642 3.364 -57 9.19328525664676 0.68386140921236 3.249 -56 9.79177729151317 0.68843571602127 3.136 -55 9.99990206550703 0.69304062008644 3.025 -54 9.80936230066492 0.69767632607103 2.916 -53 9.22775421612807 0.70234304000718 2.809 -52 8.27826469085654 0.70704096930516 2.704 -51 6.99874687593544 0.71177032276261 2.601 -50 5.4402111088937 0.71653131057379 2.5 -49 3.66479129251928 0.72132414433894 2.401 -48 1.74326781222981 0.72614903707369 2.304 -47 -0.24775425453358 0.73100620321851 2.209 -46 -2.22889914100246 0.73589585864824 2.116 -45 -4.12118485241757 0.74081822068172 2.025 -44 -5.84917192891762 0.74577350809139 1.936 -43 -7.34397097874113 0.75076194111308 1.849 -42 -8.5459890808828 0.75578374145573 1.764 -41 -9.40730556679773 0.76083913231129 1.681 -40 -9.89358246623382 0.76592833836465 1.6 -39 -9.98543345374605 0.77105158580357 1.521 -38 -9.67919672031486 0.77620910232878 1.444 -37 -8.98708095811627 0.78140111716408 1.369 -36 -7.93667863849153 0.78662786106655 1.296 -35 -6.56986598718789 0.79188956633678 1.225 -34 -4.94113351138609 0.7971864668292 1.156 -33 -3.11541363513379 0.80251879796248 1.089 -32 -1.16549204850494 0.80788679672999 1.024 -31 0.83089402817496 0.81329070171034 0.961 -30 2.79415498198926 0.81873075307798 0.9 -29 4.64602179413757 0.82420719261386 0.841 -28 6.31266637872321 0.82972026371619 0.784 -27 7.72764487555987 0.83527021141127 0.729 -26 8.83454655720153 0.84085728236435 0.676 -25 9.58924274663138 0.84648172489061 0.625 -24 9.96164608835841 0.85214378896621 0.576 -23 9.93691003633464 0.85784372623937 0.529 -22 9.51602073889516 0.86358179004156 0.484 -21 8.71575772413588 0.86935823539881 0.441 -20 7.56802495307928 0.87517331904295 0.4 -19 6.11857890942719 0.88102729942311 0.361 -18 4.42520443294852 0.88692043671716 0.324 -17 2.55541102026832 0.89285299284328 0.289 -16 0.5837414342758 0.89882523147161 0.256 -15 -1.41120008059867 0.90483741803596 0.225 -14 -3.34988150155905 0.91088981974561 0.196 -13 -5.15501371821464 0.91698270559719 0.169 -12 -6.75463180551151 0.92311634638664 0.144 -11 -8.0849640381959 0.9292910147212 0.121 -10 -9.09297426825682 0.93550698503162 0.1 -9 -9.73847630878195 0.94176453358425 0.081 -8 -9.99573603041505 0.9480639384934 0.064 -7 -9.8544972998846 0.95440547973365 0.049 -6 -9.32039085967226 0.96078943915232 0.036 -5 -8.41470984807897 0.96721610048201 0.025 -4 -7.17356090899523 0.97368574935315 0.016 -3 -5.64642473395035 0.98019867330676 0.009 -2 -3.89418342308651 0.9867551618072 0.004 -1 -1.98669330795061 0.99335550625503 0.001 0 0 1 0 1 1.98669330795061 1.00668893835402 0.001 2 3.89418342308651 1.01342261860434 0.004 3 5.64642473395035 1.02020134002676 0.009 4 7.17356090899523 1.02702540389888 0.016 5 8.41470984807897 1.03389511351357 0.025 6 9.32039085967226 1.04081077419239 0.036 7 9.8544972998846 1.04777269329916 0.049 8 9.99573603041505 1.05478118025366 0.064 9 9.73847630878195 1.06183654654536 0.081 10 9.09297426825682 1.06893910574725 0.1 11 8.0849640381959 1.07608917352979 0.121 12 6.75463180551151 1.08328706767496 0.144 13 5.15501371821464 1.09053310809034 0.169 14 3.34988150155905 1.09782761682338 0.196 15 1.41120008059867 1.10517091807565 0.225 16 -0.5837414342758 1.11256333821731 0.256 17 -2.55541102026832 1.12000520580159 0.289 18 -4.42520443294852 1.12749685157938 0.324 19 -6.11857890942719 1.13503860851394 0.361 20 -7.56802495307928 1.14263081179572 0.4 21 -8.71575772413588 1.15027379885723 0.441 22 -9.51602073889516 1.15796790938803 0.484 23 -9.93691003633464 1.16571348534986 0.529 24 -9.96164608835841 1.17351087099181 0.576 25 -9.58924274663138 1.18136041286565 0.625 26 -8.83454655720153 1.18926245984118 0.676 27 -7.72764487555987 1.19721736312181 0.729 28 -6.31266637872321 1.20522547626009 0.784 29 -4.64602179413757 1.21328715517349 0.841 30 -2.79415498198926 1.22140275816017 0.9 31 -0.83089402817496 1.22957264591493 0.961 32 1.16549204850494 1.23779718154525 1.024 33 3.11541363513379 1.24607673058738 1.089 34 4.94113351138609 1.25441166102266 1.156 35 6.56986598718789 1.2628023432938 1.225 36 7.93667863849153 1.2712491503214 1.296 37 8.98708095811627 1.2797524575205 1.369 38 9.67919672031486 1.28831264281726 1.444 39 9.98543345374605 1.29693008666577 1.521 40 9.89358246623382 1.30560517206495 1.6 41 9.40730556679773 1.31433828457558 1.681 42 8.5459890808828 1.32312981233744 1.764 43 7.34397097874113 1.33198014608653 1.849 44 5.84917192891762 1.34088967917248 1.936 45 4.12118485241757 1.349858807576 2.025 46 2.22889914100246 1.35888792992651 2.116 47 0.24775425453358 1.36797744751981 2.209 48 -1.74326781222981 1.37712776433596 2.304 49 -3.66479129251928 1.38633928705721 2.401 50 -5.4402111088937 1.39561242508609 2.5 51 -6.99874687593544 1.40494759056359 2.601 52 -8.27826469085654 1.4143451983875 2.704 53 -9.22775421612807 1.42380566623082 2.809 54 -9.80936230066492 1.43332941456034 2.916 55 -9.99990206550703 1.44291686665534 3.025 56 -9.79177729151317 1.45256844862637 3.136 57 -9.19328525664676 1.46228458943422 3.249 58 -8.22828594968708 1.47206572090898 3.364 59 -6.93525084777122 1.48191227776921 3.481 60 -5.36572918000435 1.49182469764127 3.6 61 -3.58229282236827 1.5018034210788 3.721 62 -1.65604175448309 1.51184889158225 3.844 63 0.33623047221139 1.52196155561863 3.969 64 2.31509825101539 1.53214186264135 4.096 65 4.20167036826641 1.54239026511017 4.225 66 5.92073514707224 1.55270721851134 4.356 67 7.40375889952449 1.5630931813778 4.489 68 8.59161814856497 1.57354861530962 4.624 69 9.43695669444105 1.58407398499448 4.761 70 9.9060735569487 1.59466975822832 4.9 71 9.98026652716362 1.60533640593612 5.041 72 9.65657776549277 1.61607440219289 5.184 73 8.94791172140503 1.62688422424467 5.329 74 7.88252067375316 1.63776635252977 5.476 75 6.50287840157117 1.64872127070013 5.625 76 4.86398688853798 1.6597494656428 5.776 77 3.03118356745702 1.6708514275016 5.929 78 1.07753652299442 1.68202764969889 6.084 79 -0.91906850227682 1.69327862895748 6.241 80 -2.87903316665065 1.70460486532275 6.4 81 -4.72421986398466 1.71600686218486 6.561 82 -6.3810668234795 1.72748512630109 6.724 83 -7.78352078534298 1.7390401678184 6.889 84 -8.87567033581505 1.7506725002961 7.056 85 -9.61397491879557 1.76238264072866 7.225 86 -9.96900066041596 1.77417110956869 7.396 87 -9.92659380470633 1.78603843075007 7.569 88 -9.48844497918124 1.79798513171127 7.744 89 -8.67202179485581 1.81001174341873 7.921 90 -7.50987246771676 1.82211880039051 8.1 91 -6.04832822406284 1.83430684072002 8.281 92 -4.34565622071894 1.84657640609995 8.464 93 -2.46973661736621 1.85892804184634 8.649 94 -0.49535640878367 1.87136229692281 8.836 95 1.49877209662952 1.88387972396496 9.025 96 3.43314928819899 1.89648087930495 9.216 97 5.23065765157699 1.9091663229962 9.409 98 6.81963620068136 1.92193661883829 9.604 99 8.13673737507105 1.93479233440203 9.801
0.1228420655386 1.44436790064062 4.14989619335585 2.80929954228979 3.91710443738655 -5.79285664213773 -1.49456156254977 0.36513704991097 -5.15117861415134 -1.75920446838901 -4.22973623421248 -4.1005522290024 -1.60340725271362 -2.10751676296749 -7.18503618704775 -6.76439420177965 1.19448910164725 -3.74989612977446 -0.91946785111106 -2.17992951658188 1.30251506846536 2.57061603797164 0.41389025323971 0.27115910567284 -1.42687906993845 -0.47543795638419 3.29820376088691 0.32587253884643 3.75566061883408 -0.894822326474 5.10349002762458 -2.57183182495552 2.30896888791521 -0.81627543528371 0.16835675100987 -0.21300104623407 -0.26434058469824 0.76473098312201 -0.27957183445117 0.29373315558661 -4.55662392010732 0.67960422048836 -1.90659400862799 -2.76940101762684 -1.08997297210444 2.10119466724699 -3.72339527788975 -2.79209436505247 -1.5807726203848 0.70757410531515 0.663051574136 -2.01040325352078 1.18945308394722 -0.58086460584692 0.41814829631951 -0.16391536395129 2.90876601952262 2.71112348637748 -2.16966120310971 -0.21153523310426 4.25718431990428 -1.99697994979041 0.81365930619485 3.56120984822422 0.97659909284328 -1.35644199908538 0.35712285869186 1.76851416172139 -2.88952587861019 3.3673164000231 1.40652199207335 -1.06012176807495 -0.36464505813246 -0.1243845610486 1.04175599361146 -1.82217007894793 0.1717269135954 2.1415381206607 0.40358513145925 0.06151583233444 -0.23638148633359 2.67481568207357 -0.48088524826996 -0.74728401660285 -1.12756607000052 -2.47029243868955 0.3844646328401 1.85885234998332 1.27626497243681 1.26531732465683 -1.02159531798687 -0.11991496983357 -3.07035817670374 0.32654043176716 -0.1442124330365 -2.42334213355828 0.27007627365143 2.12082601871945 -2.28079566008247 1.89803389062997 2.48215489348 -2.31493278024806 0.99818485126845 -2.72736107030692 0.81136102189444 0.97185143712749 1.28430328541426 -0.67208249573367 -0.11649447474418 -2.7451727551236 -1.81907920490784 -1.60348899889692 -0.24084322792431 -2.00082117046931 -1.9917369737084 0.22416274822465 -0.81928540531782 2.98481350443713 -1.80605500070193 1.8014054856648 1.57713888310666 -0.00325381215511 0.88577670167741 -1.03039519652981 -0.60597692982763 -0.61117825272671 0.03609754386216 3.62385748441374 0.81271566552516 1.12836658085658 1.37238261834398 -2.70968786064595 0.65179395177783 5.00094368472696 0.01965699324256 -0.21660498013829 0.15697859351866 -0.50324473023494 2.92608874436412 1.18955843588763 -0.66422101424879 -1.67138034552948 0.21532901022473 0.18150707696255 0.56342981848474 4.68587028061131 1.07281796714959 -2.12394126130281 -3.42178713663679 -3.2763139554261 0.8025968882164 -1.34305062965651 -1.73157041507151 -3.16946059684787 -3.03362237412121 -0.45049405118498 -2.77482295533373 -3.24055037225316 -4.16543303871421 -1.28502418423435 -3.70851845333193 1.89091726367211 -1.60488099470415 1.4399952037203 -1.81787666987151 2.30031814923575 3.76492682476925 2.58177049886044 2.68510371140111 5.88575909635699 4.15453965688016 6.56486099801879 3.43726100924814 8.05163602679345 7.22053041712732 4.79846919972413 1.70209750534958 -0.56093913551996 3.5915271146061 1.86801634361704 -2.36753257795678 -5.43771392560915 -5.7493408097492 -6.68722585778494 -8.02188642112283 -12.1672821356603 -13.1637806200535 -11.7441459825381 -12.6861582839036 -11.3270276465867 -9.43598176455517 -9.62475088472305 -6.27935784559543 -5.88631627082809 -0.53562988188648 -0.94028925314143 4.75892415920041 7.66685357257905 8.02746696375658 16.3247739610214
---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
