Following up the follow-up, I did some time comparisons on the 3.x
branch (after fixing a packaging error).
For full builds:
| Pre-change | Post-change
=============================|===============
Build Release | |
Tarball | 4m34s | 3m31s
---------------------------------------------
Test and | |
Package | ~20m38s | ~18m06s
---------------------------------------------
Publish | 2m38s | 1m39s
For PR builds, the time difference seems to be negligible, dominated by
the actual test run variance, but it also appears to be about 1 minute
faster by skipping the download.
-Joan "time for taquitos" Touzet
On 29/10/2020 18:09, Joan Touzet wrote:
> Following up,
>
> I've implemented a new Jenkins job that re-pulls all current couchdbdev
> images on each docker node every night. The job takes 12 minutes to run.
>
> Once a week, it also runs the "pull them all" set of images I mentioned
> a few months ago, to keep our images on Docker Hub from disappearing.
> (That run takes about an hour and happens Sunday nights, randomly,
> between 0-7h UTC.)
>
> With this we can drop the "alwaysPull true" in the Jenkinsfiles, which
> should speed up PRs and builds by a couple minutes minutes each.
>
> Those PRs are up here:
>
> https://github.com/apache/couchdb/pull/3234
> https://github.com/apache/couchdb/pull/3233
>
> which need +1s to land.
>
> That leaves a single problem scenario, namely when the couchdbdev images
> are updated and Docker stubbornly refuses to pull the latest image,
> causing build issues. In that case, simply log into our Jenkins and
> click "Build Now" here:
>
> https://ci-couchdb.apache.org/job/jenkins-cm1/job/Update%20Docker%20Containers/
>
>
> which forces the image removal and re-pull.
>
> I hope this is enough to avoid any rate limiting problems.
>
> -Joan "Gir, stop singing this instant!" Touzet
>
> On 2020-10-29 12:05 a.m., Joan Touzet wrote:
>> I just posted about this on the ASF-wide [email protected] list:
>>
>> https://lists.apache.org/thread.html/r5ccf60da8072b3c2b587152256ebaf6a0e7b81182d5e240a2b2a0f02%40%3Cbuilds.apache.org%3E
>>
>>
>> TL;DR: We're not immune, even with our build machines, and the new
>> limits kick in Monday.
>>
>> We can remove some of our workarounds for badly-cached images (at the
>> cost of lots of pain every time the couchdbci build environment image
>> changes), but on a busy day we'll probably still hit the limit.
>>
>> Let's see if ASF Infra comes through quickly with a pull-through caching
>> registry of sufficient size. If so we can make very minor tweaks to our
>> Jenkinsfiles, and go back to life as normal.
>>
>> -Joan "Doom doom, doomy doomy doom" Touzet
>>
