Currently the WebSocket tests are served over HTTP from the WebSocket server itself (which is written in Python). It looks like we can resolve the flakiness by serving the WebSocket tests from the normal Apache server that servers the rest of our HTTP tests. I'm going to work up a patch that does that. Please let me know if there's any reason we shouldn't make that change.
Thanks, Adam On Thu, Sep 30, 2010 at 3:10 PM, Adam Barth <[email protected]> wrote: > I'm investigating the websocket issue. It seems these tests are flaky > because they time out. If you know about websockets, I'd appreciate > any tips you have via #webkit. > > Adam > > > On Wed, Sep 29, 2010 at 11:31 PM, Adam Barth <[email protected]> wrote: >> Tonight I wrote a new webkit-patch command for detecting flaky tests. >> Here the tests that have flaked out on the Snow Leopard (Tests) build >> bot during the last 2000 revisions. This makes a good "hit list" of >> tests to fix to reduce flakiness. (The worst offenders are at the >> bottom). >> >> Take aways: >> >> 1) compositing/geometry/limit-layer-bounds-transformed-overflow.html >> is by far the worst offender. >> 2) The websocket tests are ridiculously flaky. >> 3) The appcache tests also have a serious flakiness problem. >> >> I'll run the last 10000 revisions over night and report on the results. >> >> Adam >> >> >> === Results === >> Occurances Test name >> 1 compositing/reflections/nested-reflection-animated.html >> 1 fast/css/font-face-download-error.html >> 1 fast/dom/collection-null-like-arguments.html >> 1 fast/history/history-subframe-with-name.html >> 1 >> fast/js/sputnik/Conformance/13_Function_Definition/13.2_Creating_Function_Objects/S13.2.2_A5_T1.html >> 1 >> fast/js/sputnik/Conformance/13_Function_Definition/13.2_Creating_Function_Objects/S13.2.2_A5_T2.html >> 1 fast/js/vardecl-preserve-arguments.html >> 1 http/tests/appcache/different-https-origin-resource-main.html >> 1 http/tests/appcache/fallback.html >> 1 http/tests/appcache/manifest-redirect.html >> 1 http/tests/appcache/origin-quota.html >> 1 http/tests/appcache/resource-redirect.html >> 1 http/tests/appcache/top-frame-3.html >> 1 http/tests/appcache/update-cache.html >> 1 http/tests/appcache/xhr-foreign-resource.html >> 1 http/tests/cache/subresource-expiration.html >> 1 http/tests/loading/basic-credentials-sent-automatically.html >> 1 http/tests/misc/uncacheable-script-repeated.html >> 1 http/tests/navigation/changing-frame-hierarchy-in-onload.html >> 1 http/tests/navigation/ping-cross-origin-from-https.html >> 1 http/tests/navigation/ping-cross-origin.html >> 1 http/tests/navigation/post-goback-same-url.html >> 1 http/tests/plugins/get-url.html >> 1 http/tests/plugins/npapi-response-headers.html >> 1 http/tests/plugins/third-party-cookie-accept-policy.html >> 1 http/tests/security/credentials-in-referer.html >> 1 >> http/tests/security/cross-frame-access-protocol-explicit-domain.html >> 1 inspector/debugger-pause-on-breakpoint.html >> 1 inspector/extensions-events.html >> 1 media/audio-constructor.html >> 1 media/video-currentTime-set.html >> 1 plugins/destroy-stream-twice.html >> 1 >> plugins/return-error-from-new-stream-doesnt-invoke-destroy-stream.html >> 1 svg/custom/use-invalid-style.svg >> 1 transitions/transition-end-event-transform.html >> 1 websocket/tests/send.html >> 1 websocket/tests/simple-stress.html >> 1 websocket/tests/sub-protocol-with-space.html >> 1 websocket/tests/url-no-trailing-slash.html >> 1 websocket/tests/url-with-empty-query.html >> 1 websocket/tests/url-with-query.html >> 1 websocket/tests/websocket-pending-activity.html >> 1 websocket/tests/workers/close-in-onmessage-crash.html >> 2 http/tests/appcache/foreign-iframe-main.html >> 2 http/tests/appcache/local-content.html >> 2 http/tests/appcache/main-resource-hash.html >> 2 http/tests/appcache/non-html.xhtml >> 2 http/tests/appcache/reload.html >> 2 http/tests/css/css-image-loading.html >> 2 http/tests/plugins/cross-frame-object-access.html >> 2 http/tests/security/cross-frame-access-port-explicit-domain.html >> 2 security/block-test.html >> 2 websocket/tests/bad-sub-protocol-non-ascii.html >> 2 websocket/tests/handshake-fail-by-sub-protocol-mismatch.html >> 2 websocket/tests/simple.html >> 2 websocket/tests/unicode.html >> 3 http/tests/appcache/cyrillic-uri.html >> 3 http/tests/appcache/deferred-events-delete-while-raising.html >> 3 http/tests/appcache/remove-cache.html >> 3 http/tests/appcache/top-frame-4.html >> 3 http/tests/navigation/image-load-in-unload-handler.html >> 3 websocket/tests/handshake-fail-by-cross-origin.html >> 3 websocket/tests/httponly-cookie.pl >> 3 websocket/tests/long-invalid-header.html >> 3 websocket/tests/sub-protocol.html >> 3 websocket/tests/url-with-query-for-no-query.html >> 4 fast/canvas/webgl/gl-object-get-calls.html >> 4 http/tests/navigation/anchor-basic.html >> 4 websocket/tests/bad-sub-protocol-control-chars.html >> 4 websocket/tests/frame-length-longer-than-buffer.html >> 4 websocket/tests/frame-length-skip.html >> 4 websocket/tests/frame-lengths.html >> 4 websocket/tests/reload-crash.html >> 4 websocket/tests/url-with-credential.html >> 5 websocket/tests/bad-sub-protocol-empty.html >> 5 websocket/tests/error-detect.html >> 5 websocket/tests/handshake-error.html >> 6 websocket/tests/handshake-challenge-randomness.html >> 6 websocket/tests/null-character.html >> 7 http/tests/security/cross-origin-css.html >> 7 websocket/tests/bufferedAmount-after-close.html >> 7 websocket/tests/close-on-unload.html >> 7 websocket/tests/frame-length-overflow.html >> 7 websocket/tests/handshake-fail-by-no-cr.html >> 8 websocket/tests/multiple-connections.html >> 9 websocket/tests/alert-in-event-handler.html >> 9 websocket/tests/close-on-unload-and-force-gc.html >> 10 websocket/tests/bad-handshake-crash.html >> 11 websocket/tests/cross-origin.html >> 12 http/tests/navigation/ping-same-origin.html >> 13 websocket/tests/close-on-unload-reference-in-parent.html >> 15 websocket/tests/close-on-navigate-new-location.html >> 19 media/audio-controls-rendering.html >> 50 compositing/geometry/limit-layer-bounds-transformed-overflow.html >> > _______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
