On Tuesday, 19 November 2019 at 13:27:31 UTC, Sebastiaan Koppe
wrote:
After having had another day where my CI can't build because of
gateway errors, this morning I had enough. (I setup an uptime
robot so we can see how bad it actually is [1], it checks every
5 min and I just started it.)
[...]
Of course, this is a bandaid solution. Probably dub can be made
to do this as well. And likely should. If it were up to me I
would go serverless.
[1] https://stats.uptimerobot.com/6mQX4Crw2L
[2] https://github.com/skoppe/dub-registry-mirror
I have updated the script to cache all routes. This means the
registry mirror at dub.bytecraft.nl will also work when the
registry at code.dlang.org is completely down, **if** your
dependencies have been cached previously.
The cache works in the following manner:
1. if a cache entry exists for a given route and it is no longer
than 5 minutes old: return that
2. if a cache entry is older than 5 minutes: fetch from upstream,
cache it and return that
3. if a cache entry is older than 5 minutes and upstream is down:
reset cache expiry and return from cache
4. if a cache entry doesn't exist: fetch from upstream, cache it
and return that
5. if a cache entry doesn't exist and upstream is down: return
error
On top of that there is a 5 second timeout on fetches from
upstream.
Essentially a cached entry is kept up-to-date (5 minute lag)
whenever upstream is alive, otherwise stale data is returned if
available.
Use it like so:
`dub build --registry=https://dub.bytecraft.nl`
Code at
https://github.com/skoppe/dub-registry-mirror/blob/master/worker.js
