On Tue, Apr 21, 2020 at 12:05 PM Ash Berlin-Taylor <[email protected]> wrote:
> I've just looked in Docker settings for it's Automated builds, and it is > possible to set up a URL that we can post to that will then trigger a > daily build. > > https://hub.docker.com/repository/registry-1.docker.io/apache/airflow/builds/05570a90-f8bf-4803-b935-f93c455ab5bb > was me testing it out (needs auth, most people won't be able to see that) > > Yes. I know this option. This problem (regular builds) and possibly triggering them via some kind of CRON job was already discussed it in detail with Daniel in https://github.com/apache/airflow/pull/8400#issuecomment-614783967 - that was PR entitle "Less frequent DockerHub Builds" which we merged already (but I am not particularly happy with this approach). Please take a look there Ash - we discussed all the options we saw at this time (including URL triggering). > So we can set up a travis job (say, since we can put encrypted info in > there. I don't think we can put secrets in our Github Actions as we > aren't admins on the repo) that would make a PSOT to this special URL > once a day, causing DockerHub to build for us. > I believe a big problem with external URL that it might be to use to DDOS our builds. And we cannot (For now) manage secrets in our Github Actions. I opened INFRA ticket and Gavin assigned it himself so likely there will be soon answered and maybe we will have a proposal from INFRA soon: https://issues.apache.org/jira/projects/INFRA/issues/INFRA-20124. If we had this possibility, URL triggered by CRON Github Action would be a possibility. We are waiting for INFRA to help with that. And I think we want to move out Travis eventually. And I do not want to add another "CRON" service just for that - it should be available to all committers to modify/fix/change and we do not want to add additional service/credentials/hidden URL secret mechanism. I think we definitely do not want to keep both GA and Travis at the same time. This is quite a bad idea to keep Travis running and complicating our toolset. Would that get us the behaviour we need without polluting our git tags? > I think I have a better solution :) See below. -ash > > On Apr 21 2020, at 10:59 am, Ash Berlin-Taylor <[email protected]> wrote: > > > What is the goal in having daily-master-ci-2020-04-21 etc docker image > > tags? When would we want to use anything than "current latest CI > > master" image? > Agree. It does clutter the namespace. And some projects are ok with that. If we do not think it might be useful we can even implement retention policy and keep only 2-3 latest tags (or even just the latest one). I think this might be a very good solution - every night when the master CRON build succeeds we delete previous "daily-master-ci-*" and create a new one with today's date. That will give us what we want, it will not clutter the namespace and additionally, we will immediately see when the last daily build succeeded. The builds in DockerHub can be triggered by regular expression for the tags so this will work. I think in this form it should all your concerns Ash (no clutter, full automation) and mine (no extra services to manage) and provides a robust solution without. Why do you think? Ash, any other concerns? Others? J.
