Hi Everyone, I want to update everyone with the further improvements I have made to new EWS. As always, please feel free to provide any feedback (either by filing bugs or contacting me directly).
New Features: - Moved following queues to from old to new EWS: • iOS 12 Tests EWS • macOS WK1 and WK2 Release Tests EWS • macOS WK1 Debug Tests EWS • GTK and WPE EWS • WinCairo EWS • Style (and watchlist) EWS - Added a new 'services' EWS to run Buildbot unit tests https://webkit.org/b/199994 - Separate status bubbles for builders and layout-testers Infrastructure Improvements: - EWS is now able to automatically recover (and retry the build) in case of certain git and network errors https://webkit.org/b/199722 - Some EWS bots now reboot periodically to ensure that start from clean state - Added KillOldProcesses step before running API or Layout tests to increase robustness https://webkit.org/b/199592 - 'view layout test results' option is now displayed next to layout-test build step https://webkit.org/b/200048 - EWS now displays pre-existing Layout test failure names in the build summary https://webkit.org/b/199941 - Machine uptime is now reported in PrintConfiguration step https://webkit.org/b/200812 - Added EWS Buildbot support to Internal scripts to easily find/reconfigure bots - Flakiness in API tests has been further reduced (thanks to multiple fixes by WebKit developers) - Added more unit tests Bug fixes: - EWS status-bubbles are sometimes multi-row with scroll-bar https://webkit.org/b/199939 - EWS: Cannot see build status page when patch is waiting for tester https://webkit.org/b/200333 - EWS: Do not append additional '(failure)' string at the end of custom failure message in EWS Buildbot https://webkit.org/b/201140 - [ews] Status bubble should display only important messages in pop-over https://webkit.org/b/201308 Only remaining queues on old EWS are windows, jsc and commit-queue, which I will be working on next. Thanks Aakash > On Jul 11, 2019, at 10:18 am, Aakash Jain <aakash_j...@apple.com> wrote: > > Hi Everyone, > > I want to update everyone with the further improvements I have made to new > EWS. As always, please feel encouraged to provide any feedback (either by > filing bugs or contacting me directly). > > New Features: > - Launched 'Security EWS' > - Added iOS-12 Builder queue on new EWS (moved from old to new EWS) > - Added WPE and GTK queues on new EWS (moved from old to new EWS) > - New EWS can now process large patches (larger than 640kb) > https://webkit.org/b/198851 > > > Infrastructure Improvements: > - EWS now automatically retries builds in case of certain infrastructure > issues and ToT compile failures https://bugs.webkit.org/show_bug.cgi?id=199025 > - Improved stability by adding KillOldProcesses step before API and Layout > tests https://webkit.org/b/199592 > - Shared bots across queues to improve efficiency https://webkit.org/b/198370 > - Added check for duplicate workers in config.json https://webkit.org/b/199240 > - Patch link now opens the pretty-patch url https://webkit.org/b/199031 > - Triggered builds should use same revision as parent build (fixes a corner > case) https://webkit.org/b/198289 > - Display pre-existing API test failures in build summary to help > bot-watchers https://webkit.org/b/199525 > - Send email notifications for failures (for maintenance/bot-watching) > https://webkit.org/b/198919 > - Remove unused buildbot tabs for better readability > https://webkit.org/b/198108 > - Allow skipping uploading built product for few builders > https://webkit.org/b/199422 > - EWS should provide option to download layout test results zip file > https://webkit.org/b/199121 > - Retry Layout test in case of failures https://webkit.org/b/199194 > - Upload test results after running layout-tests https://webkit.org/b/199120 > - Improved error message on performing certain actions (like Rebuild) which > requires authentication > - Added more unit-tests > > > Bug fixes: > - Results are clobbered in UploadTestResults and ExtractTestResults steps in > case of multiple layout test runs https://webkit.org/b/199178 > - New EWS: api-ios does not differentiate between patch compile failure and > ToT compile failure https://webkit.org/b/197850 > - Buildbot worker not pinged https://webkit.org/b/199438 > - Make PrintConfiguration platform aware https://webkit.org/b/196657 > - Status bubble should not turn orange when any build step is skipped > https://webkit.org/b/199079 > - Do not print worker environment variables in each build step > https://webkit.org/b/197319 > - Do not run unix commands for windows in PrintConfiguration > https://webkit.org/b/199605 > > Thanks > Aakash > >> On May 22, 2019, at 7:36 PM, Aakash Jain <aakash_j...@apple.com> wrote: >> >> Hi Everyone, >> >> I just wanted to update everyone with the recent improvements I have made to >> new EWS. As always, please feel encouraged to provide any feedback (either >> by filing bugs or contacting me directly). >> >> New Features: >> • EWS status-bubble now display position in queue while patch is >> waiting to be processed >> • Added webkitpy and bindings-tests EWS (moved from old to new EWS) >> • Status bubbles for webkitpy and bindings-tests EWS now display the >> exact test failures in hover-over message (https://webkit.org/b/197395, >> https://webkit.org/b/197423) >> • Added support for 'new EWS' in webkit-patch tool >> • Added 'EWS Build Archives' (similar to 'WebKit Build Archives' >> https://webkit.org/blog/7978/introducing-webkit-build-archives/). For every >> patch uploaded to Bugzilla, EWS builders build the patch for various >> platforms (currently macOS and iOS) and upload the archives to S3. These >> archives are available to download by anyone (for 14 days). The S3 URL is in >> corresponding build (e.g.: notice 'uploaded archive' link in >> https://ews-build.webkit.org/#/builders/7/builds/2477). So, if for any >> reason, you want to get a built archive for your patch, you can simply >> upload the patch to Bugzilla. (Note that if there is interest in this, we >> can enhance it further) >> >> Infrastructure Improvements: >> • Flakiness in API tests has been reduced (thanks to many WebKit >> developers) >> • Infrastructure improvements to prevent build failure due to "worker >> not pinged" (e.g.: https://ews-build.webkit.org/#/builders/9/builds/332) >> • New EWS polls bugzilla more frequently https://webkit.org/b/197138 >> • Configured DEBUG mode appropriately for Production and Development >> env https://webkit.org/b/197700 >> • Ensured that Buildbot worker logs are not lost on restarting worker >> • Do not run clean build by default on EWS builders (to improve >> efficiency) https://webkit.org/b/196897 >> • build.webkit.org and ews-build.webkit.org starting sharing code >> (although very little as of now, however the plan is to share more code) >> • Added migrations file to repository https://webkit.org/b/197729 >> • Added EWS bots information to Internal scripts to easily monitor bots >> • Added more unit-tests >> >> Bug fixes: >> • Clicking 'submit to new ews' doesn't reload status-bubble >> https://webkit.org/b/196675 >> • Clicking on white bubble navigates to page with only bubbles >> https://webkit.org/b/197520 >> • Submit to EWS buttons are not aligned properly with status-bubbles >> https://webkit.org/b/197139 >> • Status bubble should turn orange when any build step fails >> https://webkit.org/b/197812 >> • Handle bug titles with unicode characters https://webkit.org/b/196802 >> • Scripts using Buildbot API have CORS error https://webkit.org/b/196709 >> • PrintConfiguration should display Xcode version instead of SDKVersion >> https://webkit.org/b/196780 >> • Trigger queues only after uploading the archive >> https://webkit.org/b/197180 >> • Do not upload archive when Compile Fails https://webkit.org/b/196674 >> • Exception while loading status-bubble when no build step has started >> https://webkit.org/b/196676 >> • Use singular verb in failure description in case of single api test >> failure https://webkit.org/b/197013 >> • EWS should clearly indicate flaky test failures >> https://webkit.org/b/196947 >> • Use explicit imports instead of wildcard imports >> https://webkit.org/b/197194 >> • New EWS: patches on recently added queues listed as #1 for older bugs >> https://webkit.org/b/197496 >> • Improved summary text for various build steps >> >> Interesting info: Since last month, 'EWS for API tests' prevented API test >> breakage on 50+ patches >> (https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number). >> >> Thanks >> Aakash >> >>> On Apr 4, 2019, at 10:00 PM, Aakash Jain <aakash_j...@apple.com> wrote: >>> >>> Introducing brand new EWS >>> >>> with EWS for API Tests (for macOS and iOS) >>> >>> and EWS for WebKitPerl Tests! >>> >>> >>> Starting today, when you upload a patch to bugs.webkit.org, you will see >>> few more bubbles (for API tests and webkitperl tests). You might also see >>> additional button 'Submit to new EWS' (if the patch doesn't have r? flag). >>> >>> The new EWS comes with many new features and there are lot more I want to >>> add. But, I don't want you guys to wait more to start getting the benefits. >>> That's why I am rolling it out in phases, starting with EWS for API Tests >>> and WebKitPerl Tests. These are tests which are not currently covered by >>> the existing EWS. Next step would be to move queues from existing EWS to >>> new EWS one by one, with the eventual goal of moving over everything to new >>> EWS. >>> >>> >>> Why new EWS? >>> The existing EWS has certain architectural limitations. One of the >>> prominent limitation is that there is no concept of building and testing >>> the patch on different queues. If we have three queues: WK1 tests, WK2 >>> tests and API tests, all three queues would need to compile WebKit >>> independently. So WebKit would be compiled thrice instead of once. This is >>> inefficient and thereby require more hardware. >>> >>> The new EWS has separate builder and tester queues. Builder queues build >>> once and upload the archive. Multiple tester queues download that same >>> archive and run tests on that. That way WebKit is compiled only once, and >>> re-used on multiple tester queues. This improves system efficiency and >>> allows us to add new test queues with substantially less hardware. >>> >>> The new EWS uses Buildbot at the back-end, which is a production-level CI >>> system. It is easier to maintain and automatically provide various features >>> like historical build logs, real-time log streaming, easier bot management, >>> ability to retry a build etc. Plus, it’s a system most of you are already >>> familiar with (build.webkit.org). >>> >>> >>> How can you contribute: >>> If you are interested in contributing, the source code is located at: >>> ews-build (Buildbot): Tools/BuildSlaveSupport/ews-build >>> ews-app (web-app): Tools/BuildSlaveSupport/ews-app >>> >>> Detailed instructions are at: >>> https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem >>> >>> >>> Upcoming features: >>> - status-bubble should display position in queue >>> https://bugs.webkit.org/show_bug.cgi?id=196607 >>> - EWS should comment on Bugzilla bugs about failures >>> https://bugs.webkit.org/show_bug.cgi?id=196598 >>> - EWS should have a way to retry a patch >>> https://bugs.webkit.org/show_bug.cgi?id=196599 >>> - Security EWS https://bugs.webkit.org/show_bug.cgi?id=196605 >>> >>> >>> If you notice any issue, please feel free to file bugs (and assign to me). >>> >>> Thanks & Regards >>> Aakash Jain >> > _______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev