Hi there, As you might have noticed, our CI is currently red. While it’s great that we do not block our releases because of it, it’s not an ideal situation.
A short background why is broken: We use Travis CI’s virtual machines to execute our tests in combination with Sauce Labs, i.e., cordova-paramedic that runs on the Travis CI VM opens a secure tunnel to the Sauce Labs selenium grid. This allows for instance to test an iOS simulator on a Windows machine, as the simulator/emulator runs on another machine and communicates via the WebDriver protocol. Sauce Labs offers real mobile devices and virtualized ones. We only use the latter. [1] Unfortunately, there is a network issue between the Sauce Labs and the Travis CI VM’s, the tunnel seems established but no network requests come through – but not for all combos, just for some. I’ve reached out to Travis CI’s support and they pointed the finger to Sauce Labs. Multiple attempts to have a real network engineer assigned to this task instead of the service level 1 assignee failed and they’ve just closed the ticket. On top of this issue lies the problem with external contributions (i.e., outside from the Apache organization). Obviously, Sauce Labs needs some sort of credentials to work. We use the deprecated JWT addon at Travis [2], which might be disabled at any time, too. Sadly, there is no known solution on how to securely pass the credentials to external forks/pull requests. There is no solution on how to make secrets available to forks on GitHub Actions neither. As we are moving to GitHub Actions (or at least that’s our goal), I’ve taken a look how to tackle the above problem. Given the fact that we test simulators and emulators on Sauce Labs anyways, we might be able to execute everything via GitHub Actions only, i.e., no credentials issue. The GitHub Actions runner have a big variety of available Xcode and iOS versions in their virtual environments. [3] And to have some good news – iOS works! [4] – the only downside is that we can’t test iOS 11.4, but IMHO this loss would be OK. In theory, Android works, too. With the help of the reactivecircus/android-emulator-runner action we can setup the emulators with a specific target API level. It also handles accepting the permissions etc. [5] At least, the logs show that the emulator booted and some things happened (like installing the APK) For the browser tests, some changes in cordova-serve would be needed. In order to launch Chrome, Firefox etc. in headless mode, we would need to pass those arguments. Currently there is an open issue to add this functionality. [6] My plan for the next week(s) is to investigate the Android timeouts and to test if a patch in cordova-serve would be enough to make headless launching possible to make the CI green. If this works, we would have a blueprint that could be copied to the plugins. With the matrix syntax of GitHub Actions we can generate an easy to maintain and extendable testing matrix. One feature I really like is splitting the configs in different files, as opposed to the gigantic .travis.yml. Let me know what you think, I’m eager to get some feedback. My work can be seen in the feature branch feat/gh-actions [7], especially this commit: [8] Cheers, Tim [1] - https://app.saucelabs.com/open_sauce/user/snay/tests [2] - https://blog.travis-ci.com/2018-01-23-jwt-addon-is-deprecated [3] - https://github.com/actions/virtual-environments/blob/main/images/macos/macos-10.15-Readme.md#xcode [4] - https://github.com/apache/cordova-paramedic/actions/runs/385358819 [5] - https://github.com/ReactiveCircus/android-emulator-runner [6] - https://github.com/apache/cordova-serve/issues/18 [7] - https://github.com/apache/cordova-paramedic/tree/feat/gh-actions [8] - https://github.com/apache/cordova-paramedic/commit/6657551feae3840b9f4fa3a8fc23010515b5b87f -- Tim Brust, Product Engineer tim.br...@sinnerschrader.com<mailto:tim.br...@sinnerschrader.com> SinnerSchrader Deutschland GmbH | SinnerSchrader Group Part of Accenture Interactive Völckersstraße 38, 22765 Hamburg, Germany Amtsgericht Hamburg HRB-Nr. 63663 Geschäftsführung: Dr. Axel Averdung (Sprecher), Holger Blank, Kristina Bonitz, Peggy Hutchinson, Philipp Kafkoulas, Florian Langmack, Sven Schmiede, Pia Schott Büros: Berlin, Hamburg, Frankfurt a. M., München, Prag www.sinnerschrader.com | NEXT AGENCY ________________________________ This message is for the designated recipient only and may contain privileged, proprietary, or otherwise private information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of the email by you is prohibited.
