[CI] Medic Report
Hi list, Here are some updates regarding the CI this week: - Commits are now triggering builds - Periodic builds are now nightly instead of every 3 hours - cordova-vm.apache.org is now alive! @steven: please do OPIE setup to get sudo powers - Android builds are more stable; there is a JIRA for stabilizing Android’s “cordova run”: CB-9172 - PR statuses are pending, blocked on INFRA-9811 and INFRA-9678 - Android builds are now building with gradle instead of ant by default This week’s test failure rates: ~1 on Android OSX ~2 on Android Windows ~21 on iOS 0 on Windows Phone 8.1 0 on Windows 8.1 0 on Windows 8.0 0 on Windows Phone 8.0 Kindly, Dmitry P.S. Sorry I missed the update last week!
[CI] Medic Report
Hi list, Here are some points of interest regarding our CI this week: * PR-triggered and commit-triggered builds waiting on INFRA-9683 and INFRA-9678 * build-breaking issues from last week got fixed promptly * Android build turned green this week, thanks to Alexander Sorokin and others! * experimental support for automating release processes Here is this week’s closing test result status: * 0 on Android OSX (flaky builds; machine setup is suspected, and is under investigation) * 2 on Android Windows * ~21 on iOS * 0 on Windows * 0 on WP8 As always, the medic URI is: http://ci.cordova.io Kindly, Dmitry
RE: [CI] Medic Report
Jesse, friendly ping. What are your thoughts on splitting building and running to expand our test matrix? Do you think we can have build artifacts packaged somehow in a cross-platform manner? Kindly, Dmitry -Original Message- From: Dmitry Blotsky [mailto:dblot...@microsoft.com] Sent: Friday, May 22, 2015 3:57 PM To: dev@cordova.apache.org Subject: Re: [CI] Medic Report Hey Jesse, Yes, we can definitely have Medic run with different configurations. We have a full test matrix that we can set up. Currently the biggest problem is: there is a “clone and build” step that takes about 5 minutes of the time of every build, and running it for every configuration does not scale. Ideally, this step needs to only happen once per host OS (i.e. Windows and OSX), and then the respective test runs should happen. This paradigm is blocked because we’ve been unable to deal with copying/zipping/transmitting the overly long paths in the builds. I’m at a loss on how to unblock on this. Do you have any ideas? Running builds on PRs will happen as soon as INFRA-9678 and INFRA-9683 get resolved (should be next week). However, this will still lack the convenience of the nice checkmarks on GitHub (i.e. the integration is still only one-way). We’ll do that once the PRs are triggering the builds. Since we use “cordova run” to run the tests now, we support running on devices. This is a good time to plan out adding your slaves to the CI so we can use your devices. Kindly, Dmitry On May 22, 2015, at 2:26 PM, Jesse purplecabb...@gmail.com wrote: Thanks for the update Dmitry! I have kept somewhat distant from the intricacies of our CI, so I have a few questions. Do you think we can get Medic running different build variations, like with and without the --browserify flag? Can we have Medic run tests for every pull request and report results back to github so we can see if a pr breaks something? The now long ago original architecture of Medic was designed to support running tests on physical devices that would then report their results back to the url specified url in medic.json. Is this still available? And if so, can we configure test devices to report back to the federated test results server? We have nearly every device here at Adobe, but many are used rarely, we could potentially set them up for use in running tests. This would be especially useful in android land, where every device seems to 'feature' some sort of manufacturer quirk. @purplecabbage risingj.com On Fri, May 22, 2015 at 2:00 PM, Dmitry Blotsky dblot...@microsoft.com wrote: Hi list, Here are some points of interest regarding the CI: - Builds now know what commits went into them - Waiting on Infra to configure GitHub repos to send changes our way so that they will *trigger* the builds (right now they're still periodic) - Windows build recently broke, and here is the JIRA: https://issues.apache.org/jira/browse/CB-9066 - Other JIRAs outstanding, being fixed piece by piece by folks in the community Here is this week's report on test failures: - 14 on Android OSX (flaky) - ~14 on Android Windows - ~21 on iOS - Windows breaking - 0 on WP8 (flaky) As always, the Medic URI is: http://ci.cordova.io Kindly, Dmitry - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org For additional commands, e-mail: dev-h...@cordova.apache.org
Re: [CI] Medic Report
Hey Jesse, Yes, we can definitely have Medic run with different configurations. We have a full test matrix that we can set up. Currently the biggest problem is: there is a “clone and build” step that takes about 5 minutes of the time of every build, and running it for every configuration does not scale. Ideally, this step needs to only happen once per host OS (i.e. Windows and OSX), and then the respective test runs should happen. This paradigm is blocked because we’ve been unable to deal with copying/zipping/transmitting the overly long paths in the builds. I’m at a loss on how to unblock on this. Do you have any ideas? Running builds on PRs will happen as soon as INFRA-9678 and INFRA-9683 get resolved (should be next week). However, this will still lack the convenience of the nice checkmarks on GitHub (i.e. the integration is still only one-way). We’ll do that once the PRs are triggering the builds. Since we use “cordova run” to run the tests now, we support running on devices. This is a good time to plan out adding your slaves to the CI so we can use your devices. Kindly, Dmitry On May 22, 2015, at 2:26 PM, Jesse purplecabb...@gmail.com wrote: Thanks for the update Dmitry! I have kept somewhat distant from the intricacies of our CI, so I have a few questions. Do you think we can get Medic running different build variations, like with and without the --browserify flag? Can we have Medic run tests for every pull request and report results back to github so we can see if a pr breaks something? The now long ago original architecture of Medic was designed to support running tests on physical devices that would then report their results back to the url specified url in medic.json. Is this still available? And if so, can we configure test devices to report back to the federated test results server? We have nearly every device here at Adobe, but many are used rarely, we could potentially set them up for use in running tests. This would be especially useful in android land, where every device seems to 'feature' some sort of manufacturer quirk. @purplecabbage risingj.com On Fri, May 22, 2015 at 2:00 PM, Dmitry Blotsky dblot...@microsoft.com wrote: Hi list, Here are some points of interest regarding the CI: - Builds now know what commits went into them - Waiting on Infra to configure GitHub repos to send changes our way so that they will *trigger* the builds (right now they're still periodic) - Windows build recently broke, and here is the JIRA: https://issues.apache.org/jira/browse/CB-9066 - Other JIRAs outstanding, being fixed piece by piece by folks in the community Here is this week's report on test failures: - 14 on Android OSX (flaky) - ~14 on Android Windows - ~21 on iOS - Windows breaking - 0 on WP8 (flaky) As always, the Medic URI is: http://ci.cordova.io Kindly, Dmitry - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org For additional commands, e-mail: dev-h...@cordova.apache.org
[CI] Medic Report
Hi list, Here are some points of interest regarding the CI: - Builds now know what commits went into them - Waiting on Infra to configure GitHub repos to send changes our way so that they will *trigger* the builds (right now they're still periodic) - Windows build recently broke, and here is the JIRA: https://issues.apache.org/jira/browse/CB-9066 - Other JIRAs outstanding, being fixed piece by piece by folks in the community Here is this week's report on test failures: - 14 on Android OSX (flaky) - ~14 on Android Windows - ~21 on iOS - Windows breaking - 0 on WP8 (flaky) As always, the Medic URI is: http://ci.cordova.io Kindly, Dmitry
Re: [CI] Medic Report
Thanks for the update Dmitry! I have kept somewhat distant from the intricacies of our CI, so I have a few questions. Do you think we can get Medic running different build variations, like with and without the --browserify flag? Can we have Medic run tests for every pull request and report results back to github so we can see if a pr breaks something? The now long ago original architecture of Medic was designed to support running tests on physical devices that would then report their results back to the url specified url in medic.json. Is this still available? And if so, can we configure test devices to report back to the federated test results server? We have nearly every device here at Adobe, but many are used rarely, we could potentially set them up for use in running tests. This would be especially useful in android land, where every device seems to 'feature' some sort of manufacturer quirk. @purplecabbage risingj.com On Fri, May 22, 2015 at 2:00 PM, Dmitry Blotsky dblot...@microsoft.com wrote: Hi list, Here are some points of interest regarding the CI: - Builds now know what commits went into them - Waiting on Infra to configure GitHub repos to send changes our way so that they will *trigger* the builds (right now they're still periodic) - Windows build recently broke, and here is the JIRA: https://issues.apache.org/jira/browse/CB-9066 - Other JIRAs outstanding, being fixed piece by piece by folks in the community Here is this week's report on test failures: - 14 on Android OSX (flaky) - ~14 on Android Windows - ~21 on iOS - Windows breaking - 0 on WP8 (flaky) As always, the Medic URI is: http://ci.cordova.io Kindly, Dmitry
[CI] Medic Report
Hi list, Here are some points of interest regarding the CI: - Android build is flaky, and it’s an emulator + machine issue; for now, please look at the builds that finish as “Warnings” (i.e. the orange ones) - We caught and fixed two build-breaking bugs in the last two weeks - Windows Phone 8 turned green last week - We are en route to getting per-commit builds by the end of this week Here is this week’s report on test failures: Android OSX 14 Android Windows ~18 iOS 21 Windows 0 WP8 0 As always, the Medic URI is: http://ci.cordova.io Kindly, Dmitry
Re: Medic Report
That looks really useful! Is there any way to change text/background colors? -Terence Bandoian On 5/2/2015 5:29 PM, Dmitry Blotsky wrote: Hi list, Here is this week’s report on test failures: Android OSX 14 Android Windows 16 iOS 21 Windows 0 WP8 build breaking Open JIRAs: https://issues.apache.org/jira/browse/CB-8848 https://issues.apache.org/jira/browse/CB-8848 | failures iOS WP8 https://issues.apache.org/jira/browse/CB-8847 https://issues.apache.org/jira/browse/CB-8847 | failures iOS https://issues.apache.org/jira/browse/CB-8845 https://issues.apache.org/jira/browse/CB-8845 | failures Android iOS https://issues.apache.org/jira/browse/CB-8844 https://issues.apache.org/jira/browse/CB-8844 | failures Android https://issues.apache.org/jira/browse/CB-8843 https://issues.apache.org/jira/browse/CB-8843 | failures Android https://issues.apache.org/jira/browse/CB-8842 https://issues.apache.org/jira/browse/CB-8842 | failures iOS and Android I have yet to file a JIRA for the WP8 failure. Medic URI: http://ci.apache.org/waterfall?category=cordova http://ci.apache.org/waterfall?category=cordova Kindly, Dmitry - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org For additional commands, e-mail: dev-h...@cordova.apache.org
