We as committers can't change/edit secrets in GitHub I think? -- it needs a repo admin which we (commiters or PMC) are not.
Github has a solution for the redaction https://help.github.com/en/actions/reference/workflow-commands-for-github-actions#masking-a-value-in-log so we can do that with these extra secrets too. I'm about 50% through a rough draft of this solution, PR coming soon. -a On Apr 27 2020, at 12:31 pm, Jarek Potiuk <[email protected]> wrote: > BTW. I also opened the > https://issues.apache.org/jira/browse/INFRA-20197 infra ticket to > create GCP and AWS secrets for system test implementation in the > future (AIP-4). I would like to see how it works, and whether we will > be able to manage the secrets on our own easily using the built-in > secret features. Indeed it seems that other projects are using those, > so maybe that is a good-enough solution. > > In the end, we just need to be able (as committers) to be able to > change them if needed. And there is an added benefit that the secrets > are redacted from the logs automatically if printed by mistake. > https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets > > J. > > On Wed, Apr 22, 2020 at 4:12 PM Jarek Potiuk > <[email protected]> wrote: >> >> Cool. Yeah. Secrets are going to be super useful. I fully agree! >> >> On Wed, Apr 22, 2020 at 4:03 PM Ash Berlin-Taylor <[email protected]> wrote: >>> >>> Yeah I don't mind a single (well one per branch I guess) tag that we >>> re-push. >>> >>> We don't need secrets for this, but I'm working on it anyway as >>> it'll be >>> useful soon enough! >>> >>> -ash >>> >>> On Apr 21 2020, at 3:38 pm, Jarek Potiuk <[email protected]> wrote: >>> >>> > Yep pushing single 'nightly' tag works as well as expected (see the >>> > attached image). Can we settle on that solution then? >>> > >>> > Just to summarise: >>> > >>> > 1) Nightly CRON job after it is successful will update nightly >>> tags on >>> > the latest master/v1-10-test for the clarity those are the tags: >>> > nightly-master >>> > nightly-v1-10-test >>> > 2) DockerHub will be configured to update the images on tags matching >>> > `nightly-.*` regexp. It will "refresh" all the images - ci, >>> > production, production-build (we have three of them now) for each >>> > supported python version. >>> > >>> > 3) No need for any secrets. The temporary Github Token in the CI will >>> > let us move the tag to latest master/v1-10-test . This in turn will >>> > trigger DockerHyub builds. >>> > >>> > 4) Tagging will only happen if the CRON build is successful - this >>> > will make sure that whenever there is a problem that we detect early, >>> > images in Dockerhub are not affected >>> > >>> > >>> > Can we agree on that proposal? I would also love to hear what other >>> > committers think about it :). >>> > >>> > >>> > J. >>> > >>> > >>> > >>> > >>> >> On Tue, Apr 21, 2020 at 4:15 PM Jarek Potiuk >>> >> <[email protected]> wrote: >>> >> >>> >>> I am going to check the nightly tag now. If the nightly tag works >>> >>> (quite likely) - would that still be problem Ash/ Daniel? Do you >>> >>> still think that tag-triggered solution is worse in this case than >>> >>> URL-triggered one ? >>> >>> >>> >>> J. >>> >>> >>> >>> >>> >>> >>> >>>> On Tue, Apr 21, 2020 at 4:01 PM Daniel Imberman >>> >>>> <[email protected]> wrote: >>> >>>> >>> >>>>> Yeah, I'm not worried about DDOS as long as the URL is stored >>> in a >>> >>>>> secret/doesn't show up in the github action UI. >>> >>>>> >>> >>>>> >>> >>>>> On Tue, Apr 21, 2020 at 6:29 AM Ash Berlin-Taylor >>> <[email protected]> wrote: >>> >>>>> >>> >>>>>> I'm still not quite sure what problem are we solving here either...? >>> >>>>>> What is broken with the current/already merged solution? >>> >>>>>> >>> >>>>>> From a philosophical view I don't like deleting tags, and >>> this feels >>> >>>>>> like a bit of a hack to work around limitations in other systems. >>> >>>>>> (Welcome to being a developer I guess.) What you have >>> proposed is better >>> >>>>>> than having the tags build up, certainly, but I'm still not >>> wild about >>> >>>>>> it (And to check: we can't just re-push a single "nightly" >>> tag as Docker >>> >>>>>> Hub will not rebuild when a tag changes? Have we confirmed this?) >>> >>>>>> >>> >>>>>> I've read the discussion you linked to, but the only thing I see >>> >>>>>> is this >>> >>>>>> comment >>> >>>>>> https://github.com/apache/airflow/pull/8400#issuecomment-614796124 >>> >>>>>> >>> >>>>>> > But is it safe to store such URL somewhere? Is it something >>> >>>>>> that is >>> >>>>>> > sustainable long term (who will take care that it is >>> actually still >>> >>>>>> > working :)) .... Who will watch the watcher. ? >>> >>>>>> >>> >>>>>> Yes, if we store that build URL in a secure secret, for >>> instance using >>> >>>>>> the encryption approach suggested here >>> >>>>>> >>> >>>>>> https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#limits-for-secrets >>> >>>>>> we can get Apache Infra to add a single secret then we can add/change >>> >>>>>> values easily in the future. >>> >>>>>> >>> >>>>>> There is a lot of precedent in Infra tickets of creating a >>> secret for >>> >>>>>> Github Actions: >>> >>>>>> >>> >>>>>> https://issues.apache.org/jira/browse/INFRA-19602?jql=text%20~%20%22github%20actions%20secrets%22 >>> >>>>>> for example >>> >>>>>> >>> >>>>>> In the past I've used >>> https://github.com/voxpupuli/hiera-eyaml even >>> >>>>>> outside of puppet as it only encrypts the values, not the >>> whole file, >>> >>>>>> which makes it a bit easier to see what setting is changed, even >>> >>>>>> if the >>> >>>>>> setting is not visible in the diff. So what I'd suggest is we >>> ask Infra >>> >>>>>> to create an random GPG key, put the private key in a Secret >>> in Github >>> >>>>>> and then provide us with the public key. I'm happy to set >>> this up if >>> >>>>>> it's the route we want to go down. >>> >>>>>> >>> >>>>>> If it's a nightly Github action, so we'd see CRON failures as >>> we did >>> >>>>>> with Travis, no? >>> >>>>>> >>> >>>>>> -a >>> >>>>>> >>> >>>>>> >>> >>>>>> On Apr 21 2020, at 12:17 pm, Jarek Potiuk <[email protected]> >>> >>>>>> wrote: >>> >>>>>> >>> >>>>>> > On Tue, Apr 21, 2020 at 12:05 PM Ash Berlin-Taylor >>> >>>>>> > <[email protected]> >>> >>>>>> wrote: >>> >>>>>> > >>> >>>>>> >> I've just looked in Docker settings for it's Automated builds, >>> >>>>>> and it is >>> >>>>>> >> possible to set up a URL that we can post to that will then >>> >>>>>> trigger a >>> >>>>>> >> daily build. >>> >>>>>> >> >>> >>>>>> >> >>> >>>>>> https://hub.docker.com/repository/registry-1.docker.io/apache/airflow/builds/05570a90-f8bf-4803-b935-f93c455ab5bb >>> >>>>>> >> was me testing it out (needs auth, most people won't be able >>> >>>>>> to see >>> >>>>>> that) >>> >>>>>> >> >>> >>>>>> >> Yes. I know this option. This problem (regular builds) and >>> >>>>>> >> possibly >>> >>>>>> > triggering them via some kind of CRON job was already discussed >>> >>>>>> it in >>> >>>>>> > detail with Daniel in >>> >>>>>> > >>> >>>>>> >>> https://github.com/apache/airflow/pull/8400#issuecomment-614783967 - >>> >>>>>> that >>> >>>>>> > was PR entitle "Less frequent DockerHub Builds" which we >>> merged already >>> >>>>>> > (but I am not particularly happy with this approach). Please >>> >>>>>> take a look >>> >>>>>> > there Ash - we discussed all the options we saw at this time >>> >>>>>> > (including URL >>> >>>>>> > triggering). >>> >>>>>> > >>> >>>>>> > >>> >>>>>> >> So we can set up a travis job (say, since we can put encrypted >>> >>>>>> info in >>> >>>>>> >> there. I don't think we can put secrets in our Github Actions >>> >>>>>> as we >>> >>>>>> >> aren't admins on the repo) that would make a PSOT to this >>> >>>>>> special URL >>> >>>>>> >> once a day, causing DockerHub to build for us. >>> >>>>>> >> >>> >>>>>> > >>> >>>>>> > I believe a big problem with external URL that it might be to >>> >>>>>> use to DDOS >>> >>>>>> > our builds. And we cannot (For now) manage secrets in our Github >>> >>>>>> > Actions. I >>> >>>>>> > opened INFRA ticket and Gavin assigned it himself so likely >>> >>>>>> there will be >>> >>>>>> > soon answered and maybe we will have a proposal from INFRA soon: >>> >>>>>> > >>> >>>>>> >>> https://issues.apache.org/jira/projects/INFRA/issues/INFRA-20124. If >>> >>>>>> > we had >>> >>>>>> > this possibility, URL triggered by CRON Github Action would >>> be a >>> >>>>>> > possibility. We are waiting for INFRA to help with that. >>> And I >>> >>>>>> think we >>> >>>>>> > want to move out Travis eventually. And I do not want to >>> add another >>> >>>>>> "CRON" >>> >>>>>> > service just for that - it should be available to all committers >>> >>>>>> > to modify/fix/change and we do not want to add additional >>> >>>>>> > service/credentials/hidden URL secret mechanism. I think we >>> >>>>>> definitely do >>> >>>>>> > not want to keep both GA and Travis at the same time. This is >>> >>>>>> quite a bad >>> >>>>>> > idea to keep Travis running and complicating our toolset. >>> >>>>>> > >>> >>>>>> > Would that get us the behaviour we need without polluting our >>> >>>>>> git tags? >>> >>>>>> >> >>> >>>>>> > >>> >>>>>> > I think I have a better solution :) See below. >>> >>>>>> > >>> >>>>>> > -ash >>> >>>>>> >> >>> >>>>>> >> On Apr 21 2020, at 10:59 am, Ash Berlin-Taylor >>> >>>>>> <[email protected]> wrote: >>> >>>>>> >> >>> >>>>>> >> > What is the goal in having daily-master-ci-2020-04-21 etc >>> >>>>>> docker image >>> >>>>>> >> > tags? When would we want to use anything than "current >>> >>>>>> latest CI >>> >>>>>> >> > master" image? >>> >>>>>> >> >>> >>>>>> > >>> >>>>>> > Agree. It does clutter the namespace. And some projects are ok >>> >>>>>> with that. >>> >>>>>> > If we do not think it might be useful we can even implement >>> >>>>>> > retention >>> >>>>>> > policy and keep only 2-3 latest tags (or even just the latest >>> >>>>>> one). I >>> >>>>>> think >>> >>>>>> > this might be a very good solution - every night when the >>> >>>>>> master CRON >>> >>>>>> build >>> >>>>>> > succeeds we delete previous "daily-master-ci-*" and create a >>> >>>>>> new one with >>> >>>>>> > today's date. That will give us what we want, it will not >>> >>>>>> clutter the >>> >>>>>> > namespace and additionally, we will immediately see when the >>> >>>>>> last daily >>> >>>>>> > build succeeded. The builds in DockerHub can be triggered >>> by regular >>> >>>>>> > expression for the tags so this will work. >>> >>>>>> > >>> >>>>>> > I think in this form it should all your concerns Ash (no >>> >>>>>> clutter, full >>> >>>>>> > automation) and mine (no extra services to manage) and provides >>> >>>>>> a robust >>> >>>>>> > solution without. >>> >>>>>> > >>> >>>>>> > Why do you think? Ash, any other concerns? Others? >>> >>>>>> > >>> >>>>>> > J. >>> >>>>>> > >>> >>>>>> >>> >>> >>> >>> >>> >>> -- >>> >>> Jarek Potiuk >>> >>> Polidea | Principal Software Engineer >>> >>> M: +48 660 796 129 >>> >>> >>> > >>> > >>> > -- >>> > >>> > Jarek Potiuk >>> > Polidea | Principal Software Engineer >>> > M: +48 660 796 129 >> >> >> >> -- >> >> Jarek Potiuk >> Polidea | Principal Software Engineer >> >> M: +48 660 796 129 >> >> > > > -- > > Jarek Potiuk > Polidea | Principal Software Engineer > > M: +48 660 796 129 >
