Re: [JS-internals] RFC: test262 integration in jstests

2017-01-26 Thread Shu-yu Guo
Thanks for pointing this out. I was being snarky and I only use GH cursorily. On Thu, Jan 26, 2017 at 12:40 AM, Ms2ger wrote: > On 25/01/17 20:32, Shu-yu Guo wrote: > > But, like, I'm pretty sure GH's SLA is better our own... > > As someone who needs Github to do his job: it's definitely worse.

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-26 Thread Steve Fink
On 01/26/2017 12:40 AM, Ms2ger wrote: On 25/01/17 20:32, Shu-yu Guo wrote: But, like, I'm pretty sure GH's SLA is better our own... As someone who needs Github to do his job: it's definitely worse. (Also, we're not talking about just Github outages, but the union of Github outages and our own o

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-26 Thread Ms2ger
On 25/01/17 20:32, Shu-yu Guo wrote: > But, like, I'm pretty sure GH's SLA is better our own... As someone who needs Github to do his job: it's definitely worse. (Also, we're not talking about just Github outages, but the union of Github outages and our own outages.) HTH Ms2ger __

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-25 Thread Steve Fink
Hm. On a debug build, jstests for all but test262 takes 115sec; test262 takes 317sec. So we're talking about quadrupling the time -- of something that really doesn't take all that much total time at all. So if we're willing to vendor, I would suggest that we don't need any new patches at all -

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-25 Thread Shu-yu Guo
>From a CI perspective, I chatted with :gps in person in SF about this. While I personally am of the opinion of CI pulling in the external repo, :gps and release folks want to vendor the tests in. This is because they do not want our tree's open-ness to be dependent on an external service (GitHub).

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-25 Thread Tom Schuster
Thank you (and anbda of course)! I am so glad we are finally going to track test262 progress \o/. This should also make it really easy for everyone to figure out which tests are still failing. I second Shu's opinion on defaults and excluding directories. What is the process of keeping test262 up-

Re: [JS-internals] RFC: test262 integration in jstests

2017-01-25 Thread Shu-yu Guo
Woohoo, thanks for the CI work! Outside of CI, I am strongly in favor jstests.py running test262 by default. Concretely, I am in favor of: - Default set to be everything, including test262, and - Ability to exclude entire subdirectories They supersede many of our tests and is the general source

[JS-internals] RFC: test262 integration in jstests

2017-01-25 Thread Steve Fink
anba has done some major work to get test262 runnable as a CI test, and I've been looking into creating taskcluster jobs for it. One thing that could use input from a wider audience is how we want it to work with running jstests manually. The tests replace the js/src/tests/test262 directory, wh