Re: [Chicken-users] spock tests
On 2012-09-08, at 4:54 PM, Felix fe...@call-with-current-continuation.org wrote: In my experiments (updated since we last talked to the most current version of Chrome, Firefox and Safari) I have found that the performance of Spock (and Scheme2JS) varies a lot between JavaScript VMs. For example Nitro (Safari 6.0) runs JS code generated by Spock about 20 to 30 times slower than V8 (Chrome 21.0.1180.89). I have attached below the table of results from my Scheme workshop paper. The programs ctak, contfib30, btsearch2000 and threads10 use continuations heavily. Very good! Thanks or the timing results, this is certainly very interesting and I'm happy that Spock performs so well, considering that it uses an implementation strategy that brings out the worst in JS engines. BTW, did you use the -optimize switch? I did not use the -optimize switch for the experiments reported previously. The reasons are explained in the Scheme workshop paper (which by the way is now available here: http://users-cs.au.dk/danvy/sfp12/programme-sfp12.html). I just ran the experiment on V8 with Spock, this time using the -optimize switch. The results are: Execution times using V8 (Chrome 21.0.1180.89): Gambit-JSScheme2JS Spock fib35 .801.541.9x2.40 3.0x nqueens12 .72 .761.1x2.34 3.3x oddeven .831.922.3x5.62 6.8x ctak.18 17.64 95.9x .66 3.6x contfib30 1.17 106.01 90.9x3.60 3.1x btsearch2000 1.35 25.40 18.8x9.28 6.9x threads10 1.34 24.68 18.5x4.71 3.5x So the -optimize switch has actually slowed down the execution on the last 4 programs (those which use call/cc). Can you explain what the -optimize switch does? Note that these experiments were done with a previous release of Spock (the one that was current on July 6... I don't know how to get the Spock version number from chicken-spock). I know that you have released Spock 0.9 but I had problems installing the new version on my OS X 10.8.1 computer so I can't say if there is an improvement in the latest version. Marc ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
The threads test started showing me black swirls on a pink background, that grew in synch together, then started again, but after a few hours it slowly started to change into a glittering tunnel of rainbow light and I started to hear voices whispering at me in an unfamiliar language. Is this normal? Yes, that absolutely normal - it's a hidden feature. Wait until you see the bats. cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
I just ran the experiment on V8 with Spock, this time using the -optimize switch. The results are: Execution times using V8 (Chrome 21.0.1180.89): Gambit-JSScheme2JS Spock fib35 .801.541.9x2.40 3.0x nqueens12 .72 .761.1x2.34 3.3x oddeven .831.922.3x5.62 6.8x ctak.18 17.64 95.9x .66 3.6x contfib30 1.17 106.01 90.9x3.60 3.1x btsearch2000 1.35 25.40 18.8x9.28 6.9x threads10 1.34 24.68 18.5x4.71 3.5x So the -optimize switch has actually slowed down the execution on the last 4 programs (those which use call/cc). Can you explain what the -optimize switch does? Well, it (normally) optimizes the code. It enables things like copy-propagation and dead-code elimination, which I assume are enabled by default in Gambit. I can't explain the slowdown. There are too many levels involved when compiling to JS... Note that these experiments were done with a previous release of Spock (the one that was current on July 6... I don't know how to get the Spock version number from chicken-spock). I know that you have released Spock 0.9 but I had problems installing the new version on my OS X 10.8.1 computer so I can't say if there is an improvement in the latest version. I assume there isn't. The simple tests I did ran at about the same speed, but this may be different for your tests or continuation-intensive things. cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
whoa On 9/9/12, Alaric Snell-Pym ala...@snell-pym.org.uk wrote: The threads test started showing me black swirls on a pink background, that grew in synch together, then started again, but after a few hours it slowly started to change into a glittering tunnel of rainbow light and I started to hear voices whispering at me in an unfamiliar language. Is this normal? it might be. are you currently living in an institution of some sort? also, if the language turns out to be java, you should be worried- deeply worried ... live long and prosper! [someone had to say it] -- \js [http://or8.net/~johns/] : complete obscure contrariness ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On 09/09/12 15:56, john saylor wrote: also, if the language turns out to be java, you should be worried- deeply worried ... It was saying something like Ph'nglui mglw'nafh Riastradh R'lyeh wgah'nagl fhtagn ABS -- Alaric Snell-Pym http://www.snell-pym.org.uk/alaric/ ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Alaric Snell-Pym ala...@snell-pym.org.uk writes: On 09/09/12 15:56, john saylor wrote: also, if the language turns out to be java, you should be worried- deeply worried ... It was saying something like Ph'nglui mglw'nafh Riastradh R'lyeh wgah'nagl fhtagn Did the program you transform, by a chance, contain 'silver as the car of one of the *gates* alist pairs? That'd probably explain it all. -- FSF associate member #7257 http://sfd.am-1.org/ ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Hi, The threads test started showing me black swirls on a pink background, that grew in synch together, then started again, but after a few hours it slowly started to change into a glittering tunnel of rainbow light and I started to hear voices whispering at me in an unfamiliar language. Is this normal? Yes, that absolutely normal - it's a hidden feature. Wait until you see the bats. I think there must be a bug in my browser. I'm on Firefox 10.0.7 on Debian Wheezy. I've sat here all afternoon watching the swirls and I've not seen the rainbow tunnel yet. I also didn't hear any sounds from my speakers. How long does it usually take? Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On 9/9/12, Andy Bennett andy...@ashurst.eu.org wrote: How long does it usually take? how long do you have? -- \js [http://or8.net/~johns/] : complete obscure contrariness ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Hi, How long does it usually take? how long do you have? I can run it tomorrow evening when I get home from work. Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Hey, folks! Many, many thanks for testing so pervasively. This is indeed good news, as it shows that the stuff generally works. As you all reported, some test-cases fail, but this is known and caused by minor bugs or R5RS incompatibilities that result from the way Scheme is mapped onto JS semantics in this translator. Now that we know that Spock is basically working, I can continue improving it. The performance is not that great but sufficient (and in fact doesn't look too bad when compared with Scheme2JS or Gambit's forthcoming JS backend, in particular when continuations are used heavily). Also, Spock is more geared towards dynamic code generation than for doing ahead-of-time compilation of heavyweight programs. Thanks again for your assistance! cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Hi, I have Iceweasel (Firefox) 10.0.7 from Debian Wheezy (Testing). http://www.call-with-current-continuation.org/spock/test.html (eq? car car) FAIL (not (eq? (quote bitBlt) (string-symbol bitBlt))) FAIL (not (string-number 1e3 16)) FAIL tests succeded: 204 tests failed: 3 total number of tests: 207 http://www.call-with-current-continuation.org/spock/threads.html It repeatedly prints 9 swirls on a pink background. Some of the swirls go in different directions. The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). Good work! Thread support is certainly cool. Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On 08/09/12 14:49, Andy Bennett wrote: Hi, I have Iceweasel (Firefox) 10.0.7 from Debian Wheezy (Testing). http://www.call-with-current-continuation.org/spock/test.html (eq? car car) FAIL (not (eq? (quote bitBlt) (string-symbol bitBlt))) FAIL (not (string-number 1e3 16)) FAIL tests succeded: 204 tests failed: 3 total number of tests: 207 I too have suffered some inattention to details: (string-symbol #t) == #t BUT EXPECTED #f (#procedure 34 5 7 38 6) == NaN BUT EXPECTED 38 Map is call/cc safe, but probably not tail recursive or inefficient. (every-of #t) = #f ; *** wrong ***, desired result: = #t correct examples : 156 wrong examples : 1 Regards, @ndy -- andy...@ashurst.eu.org http://www.ashurst.eu.org/ 0x7EBA75FF ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On 2012-09-08, at 8:28 AM, Felix fe...@call-with-current-continuation.org wrote: Hey, folks! Many, many thanks for testing so pervasively. This is indeed good news, as it shows that the stuff generally works. As you all reported, some test-cases fail, but this is known and caused by minor bugs or R5RS incompatibilities that result from the way Scheme is mapped onto JS semantics in this translator. Now that we know that Spock is basically working, I can continue improving it. The performance is not that great but sufficient (and in fact doesn't look too bad when compared with Scheme2JS or Gambit's forthcoming JS backend, in particular when continuations are used heavily). Also, Spock is more geared towards dynamic code generation than for doing ahead-of-time compilation of heavyweight programs. Thanks again for your assistance! cheers, felix In my experiments (updated since we last talked to the most current version of Chrome, Firefox and Safari) I have found that the performance of Spock (and Scheme2JS) varies a lot between JavaScript VMs. For example Nitro (Safari 6.0) runs JS code generated by Spock about 20 to 30 times slower than V8 (Chrome 21.0.1180.89). I have attached below the table of results from my Scheme workshop paper. The programs ctak, contfib30, btsearch2000 and threads10 use continuations heavily. The paper will be available on the Scheme workshop web page. Marc Execution times using V8 (Chrome 21.0.1180.89): Gambit-JSScheme2JS Spock fib35 .801.541.9x2.40 3.0x nqueens12 .72 .761.1x2.33 3.3x oddeven .831.922.3x5.62 6.8x ctak.18 17.64 95.9x .59 3.2x contfib30 1.17 106.01 90.9x3.38 2.9x btsearch2000 1.35 25.40 18.8x7.93 5.9x threads10 1.34 24.68 18.5x4.40 3.3x Execution times using JagerMonkey (Firefox 15.0.1): Gambit-JSScheme2JS Spock fib35 1.077.667.2x 20.73 19.4x nqueens12 1.492.621.8x 12.97 8.7x oddeven1.091.931.8x 33.91 31.1x ctak 1.25 30.38 24.3x1.86 1.5x contfib30 5.88 156.79 26.7x 10.61 1.8x btsearch2000 11.19 27.162.4x 16.54 1.5x threads10 6.97 34.364.9x 22.11 3.2x Execution times using Nitro (Safari 6.0): Gambit-JSScheme2JS Spock fib35 1.326.534.9x 84.69 64.2x nqueens12 1.311.891.4x 45.15 34.4x oddeven2.521.55 .6x 143.97 57.1x ctak.30 52.88 177.4x4.18 14.0x contfib30 1.60 2311.69 1443.0x 25.98 16.2x btsearch2000 3.46 102.98 29.7x 92.02 26.6x threads10 3.03 74.21 24.5x 74.88 24.7x ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
In my experiments (updated since we last talked to the most current version of Chrome, Firefox and Safari) I have found that the performance of Spock (and Scheme2JS) varies a lot between JavaScript VMs. For example Nitro (Safari 6.0) runs JS code generated by Spock about 20 to 30 times slower than V8 (Chrome 21.0.1180.89). I have attached below the table of results from my Scheme workshop paper. The programs ctak, contfib30, btsearch2000 and threads10 use continuations heavily. Very good! Thanks or the timing results, this is certainly very interesting and I'm happy that Spock performs so well, considering that it uses an implementation strategy that brings out the worst in JS engines. BTW, did you use the -optimize switch? cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Both are fairly swift for me. OSX, Chrome Version 21.0.1180.89 -Dan On Fri, Sep 7, 2012 at 3:01 PM, Felix fe...@call-with-current-continuation.org wrote: Hello! There has been a new version of Spock (0.9) (that Scheme-JS compiler thingy) that uses a slightly changed unwinding strategy (return instead of throw), which promises better compatibility with stupid browsers and flaky JS engines. If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html and http://www.call-with-current-continuation.org/spock/threads.html The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Felix scripsit: If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html Works fast under Chrome 22.0.1229.14 beta-m, slowly (with a warning dialogue box) under IE 7.0.5730.13, both running on Windows XP SP3. http://www.call-with-current-continuation.org/spock/threads.html Works under Chrome, not under IE. -- The first thing you learn in a lawin' familyJohn Cowan is that there ain't no definite answers co...@ccil.org to anything. --Calpurnia in To Kill A Mockingbird ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
All good on Chrome 21.0.1180.79 and Firefox 14, both on Linux, except for (every-of #t) = #f ; *** wrong ***, desired result: = #t Speed is OK too. On 8 September 2012 00:01, Felix fe...@call-with-current-continuation.org wrote: Hello! There has been a new version of Spock (0.9) (that Scheme-JS compiler thingy) that uses a slightly changed unwinding strategy (return instead of throw), which promises better compatibility with stupid browsers and flaky JS engines. If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html and http://www.call-with-current-continuation.org/spock/threads.html The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
And it's the same on Safari 6, OSX 10.7 On 8 September 2012 00:21, Shawn Rutledge shawn.t.rutle...@gmail.com wrote: All good on Chrome 21.0.1180.79 and Firefox 14, both on Linux, except for (every-of #t) = #f ; *** wrong ***, desired result: = #t Speed is OK too. On 8 September 2012 00:01, Felix fe...@call-with-current-continuation.org wrote: Hello! There has been a new version of Spock (0.9) (that Scheme-JS compiler thingy) that uses a slightly changed unwinding strategy (return instead of throw), which promises better compatibility with stupid browsers and flaky JS engines. If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html and http://www.call-with-current-continuation.org/spock/threads.html The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). cheers, felix ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
Hi Felix, On Sat, 08 Sep 2012 00:01:20 +0200 (CEST) Felix fe...@call-with-current-continuation.org wrote: There has been a new version of Spock (0.9) (that Scheme-JS compiler thingy) that uses a slightly changed unwinding strategy (return instead of throw), which promises better compatibility with stupid browsers and flaky JS engines. Very nice! If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html and http://www.call-with-current-continuation.org/spock/threads.html The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). I run the tests on the following browsers: - Chromium 18.0.1025.168 - Firefox 15 - IE 6 - IE 8 For test.html, I get (same results on all browsers): correct examples : 156 wrong examples : 1 I think the wrong example is: (string-symbol #t) == #t BUT EXPECTED #f On all browsers I get a warning like: ReferenceError: 25min is not defined threads.html run ok on Chromium and Firefox. The IE versions I have here don't support canvas, so that test could not be run. Best wishes. Mario -- http://parenteses.org/mario ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On Fri, 07 Sep 2012 18:18:05 -0400 Mario Domenech Goulart mario.goul...@gmail.com wrote: On Sat, 08 Sep 2012 00:01:20 +0200 (CEST) Felix fe...@call-with-current-continuation.org wrote: There has been a new version of Spock (0.9) (that Scheme-JS compiler thingy) that uses a slightly changed unwinding strategy (return instead of throw), which promises better compatibility with stupid browsers and flaky JS engines. Very nice! If you have a minute, please point your browser to http://www.call-with-current-continuation.org/spock/test.html and http://www.call-with-current-continuation.org/spock/threads.html The former takes quite a while to run, the latter needs canvas support. I tested with Conkeror 0.9.1 and Firefox 3.6.3, which seem to run both tests OK. I would be very interested to see whether these work or fail for you, and on which browsers (IE in particular). I run the tests on the following browsers: - Chromium 18.0.1025.168 - Firefox 15 - IE 6 - IE 8 For test.html, I get (same results on all browsers): correct examples : 156 wrong examples : 1 I think the wrong example is: (string-symbol #t) == #t BUT EXPECTED #f On all browsers I get a warning like: ReferenceError: 25min is not defined Hmmm. Shawn mentioned that he gets (every-of #t) = #f ; *** wrong ***, desired result: = #t I get that too, but since the test reported one wrong result and I saw the string-symbol failure, I stopped there. So, it seems that there are three problems: - (every-of #t) = #f - (string-symbol #t) = #t - the wrong examples counter. :-) Shouldn't (string-symbol #t) raise an error? BTW, I used Chromium and Firefox on Linux. Best wishes. Mario -- http://parenteses.org/mario ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On 8 September 2012 00:26, Mario Domenech Goulart mario.goul...@gmail.com wrote: On Fri, 07 Sep 2012 18:18:05 -0400 Mario Domenech Goulart mario.goul...@gmail.com wrote: I think the wrong example is: (string-symbol #t) == #t BUT EXPECTED #f On all browsers I get a warning like: ReferenceError: 25min is not defined Oops I get those too. After seeing 1 wrong I searched for wrong and stopped there. ;-) Hmmm. Shawn mentioned that he gets (every-of #t) = #f ; *** wrong ***, desired result: = #t I get that too, but since the test reported one wrong result and I saw the string-symbol failure, I stopped there. So, it seems that there are three problems: - (every-of #t) = #f - (string-symbol #t) = #t - the wrong examples counter. :-) Shouldn't (string-symbol #t) raise an error? BTW, I used Chromium and Firefox on Linux. ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] spock tests
On Sat, 8 Sep 2012 00:40:46 +0200 Shawn Rutledge shawn.t.rutle...@gmail.com wrote: On 8 September 2012 00:26, Mario Domenech Goulart mario.goul...@gmail.com wrote: On Fri, 07 Sep 2012 18:18:05 -0400 Mario Domenech Goulart mario.goul...@gmail.com wrote: I think the wrong example is: (string-symbol #t) == #t BUT EXPECTED #f On all browsers I get a warning like: ReferenceError: 25min is not defined Oops I get those too. After seeing 1 wrong I searched for wrong and stopped there. ;-) Now I see the tests are divided into sections. The one which contains 156 correct examples and one wrong is the last. I get the following failures: (not (eq? (quote bitBlt) (string-symbol bitBlt))) FAIL (not (string-number 1e3 16)) FAIL (eq? car car) FAIL (string-symbol #t) == #t BUT EXPECTED #f (#procedure 34 5 7 38 6) == NaN BUT EXPECTED 38 (every-of #t) = #f ; *** wrong ***, desired result: = #t (Just checked Firefox and Chromium this time) Best wishes. Mario -- http://parenteses.org/mario ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
