Re: [webkit-dev] Running W3C tests using wptserve
Humm, the missing folder should have been created by AutoInstaller. Can it be removed by some clean-up between the runs? I will shortly write a fix for that particular issue but this will probably not solve the problem if the whole module is missing. y 2015-02-01 20:10 GMT+01:00 Alexey Proskuryakov : > > This issue occurs on Mac bots. Windows EWS does not run regression tests, so > it would not be affected. > > Here is the log output: > > WARNING:web-platform-test-launcher:/Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test/__init__.py > is not present, creating it as empty file > Traceback (most recent call last): > File > "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", > line 21, in > create_wpt_empty_file_if_needed(['tools', 'pywebsocket', 'src', 'test', > '__init__.py']) > File > "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", > line 17, in create_wpt_empty_file_if_needed > open(file_path, 'a').close() > IOError: [Errno 2] No such file or directory: > '/Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test/__init__.py' > > Looks like the directory > /Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test > does not exist. > > - Alexey > >> 1 февр. 2015 г., в 0:27, youenn fablet написал(а): >> >> Looking at the log, the dependencies seem to be installed correctly. >> Is it possible to get access to the corresponding >> WebKitBuild/Release/layout-test-results, in particular >> wptwk_process_log.out.txt? >> >> Also, does it happen on all bots or only windows bot? >> >> >> 2015-02-01 4:41 GMT+01:00 Alexey Proskuryakov : >>> 31 янв. 2015 г., в 11:57, youenn fablet написал(а): Currently, only two tests are run within that folder: - web-platform-tests/domparsing/DOMParser-parseFromString-html.html - web-platform-tests/domparsing/insert-adjacent.html Should there be any issus with those tests, the problem may be related to running wptserve. >>> >>> I see that these tests fail on EWS, any suggestions for how to debug the >>> issue? The EWS setup is not much different from regular bots - the biggest >>> difference is that EWS machines have more cores. >>> >>> https://webkit-queues.appspot.com/results/6309998137180160 >>> >>> - Alexey >>> > > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Running W3C tests using wptserve
This issue occurs on Mac bots. Windows EWS does not run regression tests, so it would not be affected. Here is the log output: WARNING:web-platform-test-launcher:/Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test/__init__.py is not present, creating it as empty file Traceback (most recent call last): File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 21, in create_wpt_empty_file_if_needed(['tools', 'pywebsocket', 'src', 'test', '__init__.py']) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 17, in create_wpt_empty_file_if_needed open(file_path, 'a').close() IOError: [Errno 2] No such file or directory: '/Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test/__init__.py' Looks like the directory /Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/src/test does not exist. - Alexey > 1 февр. 2015 г., в 0:27, youenn fablet написал(а): > > Looking at the log, the dependencies seem to be installed correctly. > Is it possible to get access to the corresponding > WebKitBuild/Release/layout-test-results, in particular > wptwk_process_log.out.txt? > > Also, does it happen on all bots or only windows bot? > > > 2015-02-01 4:41 GMT+01:00 Alexey Proskuryakov : >> >>> 31 янв. 2015 г., в 11:57, youenn fablet написал(а): >>> >>> Currently, only two tests are run within that folder: >>> - web-platform-tests/domparsing/DOMParser-parseFromString-html.html >>> - web-platform-tests/domparsing/insert-adjacent.html >>> >>> Should there be any issus with those tests, the problem may be related >>> to running wptserve. >> >> I see that these tests fail on EWS, any suggestions for how to debug the >> issue? The EWS setup is not much different from regular bots - the biggest >> difference is that EWS machines have more cores. >> >> https://webkit-queues.appspot.com/results/6309998137180160 >> >> - Alexey >> ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Running W3C tests using wptserve
2015-02-01 3:08 GMT+01:00 Benjamin Poulain : > Hi Youenn, > > Thanks for working on this. > > How do we run the imported tests? Do we just use run-webkit-test passing > 'imported/w3c'? Yes. > Is there a way to get the percentage of W3C tests that succeed? It would be > useful to have a report per directory so that we improve it over time until > we reach 100%. That is part of another tool: https://bugs.webkit.org/show_bug.cgi?id=134766 It basically uses run-webkit-test on the whole W3C test suite (or a subset). It checks conformance by searching for PASS/FAIL assertions. You can find an old version of the report here: http://youennf.github.io/w3c-reports/12092014/w3c_conformance_results.html That tool would also be used to discriminate which tests to import. https://bugs.webkit.org/show_bug.cgi?id=134767 would then be used to refresh LayoutTests/imported/w3c (new tests, -expected.txt creation/refresh) I plan to write a page on the wiki when these things will start to appear on the trunk. Any suggestion/help most welcome. > > Benjamin > > > On 1/31/15 11:57 AM, youenn fablet wrote: >> >> Hi all, >> >> W3C wptserve server is now used to serve all tests under >> LayoutTests/imported/w3c/web-platform-tests. >> >> Currently, only two tests are run within that folder: >> - web-platform-tests/domparsing/DOMParser-parseFromString-html.html >> - web-platform-tests/domparsing/insert-adjacent.html >> >> Should there be any issus with those tests, the problem may be related >> to running wptserve. >> The wptserve server log is dumped in the the layout-test-results folder. >> Please let me know if you see any such issue, particularly on WebKit bots. >> >> Next planned steps in that area are: >> 1. Update Tools/Scripts/import-w3c-tests according this new set-up >> 2. Migrate already imported W3C WPT tests within >> LayoutTests/imported/w3c/web-platform-tests >> 3. Further improve the tools to make import of new tests and re-import >> existing tests as easy as possible >> >> Any suggestion welcome. >> >> Thanks, >> youenn >> ___ >> webkit-dev mailing list >> webkit-dev@lists.webkit.org >> https://lists.webkit.org/mailman/listinfo/webkit-dev > > > ___ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Running W3C tests using wptserve
Looking at the log, the dependencies seem to be installed correctly. Is it possible to get access to the corresponding WebKitBuild/Release/layout-test-results, in particular wptwk_process_log.out.txt? Also, does it happen on all bots or only windows bot? 2015-02-01 4:41 GMT+01:00 Alexey Proskuryakov : > >> 31 янв. 2015 г., в 11:57, youenn fablet написал(а): >> >> Currently, only two tests are run within that folder: >> - web-platform-tests/domparsing/DOMParser-parseFromString-html.html >> - web-platform-tests/domparsing/insert-adjacent.html >> >> Should there be any issus with those tests, the problem may be related >> to running wptserve. > > I see that these tests fail on EWS, any suggestions for how to debug the > issue? The EWS setup is not much different from regular bots - the biggest > difference is that EWS machines have more cores. > > https://webkit-queues.appspot.com/results/6309998137180160 > > - Alexey > ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Running W3C tests using wptserve
> 31 янв. 2015 г., в 11:57, youenn fablet написал(а): > > Currently, only two tests are run within that folder: > - web-platform-tests/domparsing/DOMParser-parseFromString-html.html > - web-platform-tests/domparsing/insert-adjacent.html > > Should there be any issus with those tests, the problem may be related > to running wptserve. I see that these tests fail on EWS, any suggestions for how to debug the issue? The EWS setup is not much different from regular bots - the biggest difference is that EWS machines have more cores. https://webkit-queues.appspot.com/results/6309998137180160 - Alexey ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] Running W3C tests using wptserve
Hi Youenn, Thanks for working on this. How do we run the imported tests? Do we just use run-webkit-test passing 'imported/w3c'? Is there a way to get the percentage of W3C tests that succeed? It would be useful to have a report per directory so that we improve it over time until we reach 100%. Benjamin On 1/31/15 11:57 AM, youenn fablet wrote: Hi all, W3C wptserve server is now used to serve all tests under LayoutTests/imported/w3c/web-platform-tests. Currently, only two tests are run within that folder: - web-platform-tests/domparsing/DOMParser-parseFromString-html.html - web-platform-tests/domparsing/insert-adjacent.html Should there be any issus with those tests, the problem may be related to running wptserve. The wptserve server log is dumped in the the layout-test-results folder. Please let me know if you see any such issue, particularly on WebKit bots. Next planned steps in that area are: 1. Update Tools/Scripts/import-w3c-tests according this new set-up 2. Migrate already imported W3C WPT tests within LayoutTests/imported/w3c/web-platform-tests 3. Further improve the tools to make import of new tests and re-import existing tests as easy as possible Any suggestion welcome. Thanks, youenn ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] Running W3C tests using wptserve
Hi all, W3C wptserve server is now used to serve all tests under LayoutTests/imported/w3c/web-platform-tests. Currently, only two tests are run within that folder: - web-platform-tests/domparsing/DOMParser-parseFromString-html.html - web-platform-tests/domparsing/insert-adjacent.html Should there be any issus with those tests, the problem may be related to running wptserve. The wptserve server log is dumped in the the layout-test-results folder. Please let me know if you see any such issue, particularly on WebKit bots. Next planned steps in that area are: 1. Update Tools/Scripts/import-w3c-tests according this new set-up 2. Migrate already imported W3C WPT tests within LayoutTests/imported/w3c/web-platform-tests 3. Further improve the tools to make import of new tests and re-import existing tests as easy as possible Any suggestion welcome. Thanks, youenn ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
