Hopefully last set of questions for now... 1) It sounds like there is a risk that as the ASF grows, GH may not be able to grow with us. Did I understand that correctly? 2) If we have money to offer GH, why can't we offer money to the CI Vendors so we aren't really abusing their free tiers? 3) Does GH track my activity in the ASF GH repos as part of the API usage for Apache? IOW, am I adding to the ASF API count by closing an issue on github.com? Or if I ran a script on my computer that closed the issue by using their API?
I think builds.a.o is a great free service, but AIUI, the no-third-party-write-access rule is independent of whether CI is free or not. I cannot pay money and get write-access to the ASF repos. So I think I'm trying to see if there is a solution even if it did cost money. Thanks in advance, -Alex On 2/3/20, 7:03 PM, "David Nalley" <[email protected]> wrote: On Tue, Feb 4, 2020 at 3:56 AM Alex Harui <[email protected]> wrote: > > Some questions inline. Apologies in advance for not really understanding this stuff. I'm primarily a client-side developer. My projects do not have automated PR testing at this point in time. I'm mainly exploring in case we become popular enough some day to need it. > > My line of thinking is that MS has, at least for now, generously provided free Azure VMs to ASF committers. If N committers from a project each get a VM, run CI on it, figure out some way to distribute PRs to those VMs, is there a viable workflow? > > On 2/3/20, 6:38 PM, "David Nalley" <[email protected]> wrote: > > Hi Alex, > > So this was explored. It creates some problems - first double the > administration overhead - most of that is automated, but it means that > our API usage doubles, and we're already hitting limits from Github. > > Is that a max-traffic limit or a limit on traffic before we have to start paying for usage? Max number of calls - and we've tried offering up money, they don't offer a product with more API calls. Greg has even raised this issue all the way to the CEO of Github. > > Second - at least one CI vendor thanked us for not doing that exactly > - because the 'best' way to do it is to create an org per project or > org per repo - and then the free tier is dedicated to that org. Except > that's essentially abusing their free tier. > > Is "best" defined as lowest cost to the CI vendor or something else? What would the "second-best" scenario look like if there is one? Best - well it's the cheapest for us, and it gives the most control to the projects. So great from that perspective, but likely a bit unethical and abusive. It's essentially abusing all of the CI vendors generosity by horizontally scaling our consumption of their freebies and using them per-repo or per project instead of per organization. > > Finally - from a practical perspective, if everyone submits PRs and > does testing against this apacheci org - that has become the de facto > repo - it's where everyone is doing their work, and it makes > provenance tracking. > > Didn't the ASF have read-only mirrors of repos? I think it led to some confusion, but I think folks still figured out. > Not anymore. We have an active-active copy of the repositories. People can actively commit against either our repos or the GH repos, and we magically move commits between the two. (There's an upcoming blog post on how all of this magic works) > As an aside - the mandate for no write access is not an infrastructure > policy, it's a legal affairs requirement - we're merely implementing > it. > > --David > > On Tue, Feb 4, 2020 at 3:24 AM Alex Harui <[email protected]> wrote: > > > > Moving board@ to BCC. Attempting to move discussion to builds@ > > > > I’m fine with the ASF maintaining its position on stricter provenance and therefore disallowing third-party write-access to repos. > > > > A suggestion was made, if I understood it correctly, to create a whole other set of repos that could be written to by third-parties. Would such a thing work? Then a committer would have to manually bring commits back from that other set to the canonical repo. That seems viable to me. > > > > A concern was raised that the project might cut its release from the “other set”, but IMO, that would be ok if the release artifacts could be verified, which should be possible by comparing the canonical repo against the “other repo”, at least for the source package, and if there are reproducible binaries, for the binary artifacts as well. > > > > Thoughts? > > -Alex > > > > From: Greg Stein <[email protected]> > > Reply-To: "[email protected]" <[email protected]> > > Date: Monday, February 3, 2020 at 5:17 PM > > To: "[email protected]" <[email protected]> > > Subject: Re: [CI] What are the troubles projects face with CI and Infra > > > > On Mon, Feb 3, 2020 at 6:48 PM Alex Harui <[email protected]<mailto:[email protected]>> wrote: > > >... > > How does Google or other non-ASF open source projects manage the provenance tracking? > > > > Note that most F/OSS projects don't worry about provenance to the level the Foundation worries. That affords them some flexibility that our choices do not allow. Those projects may also choose to trust tools with write access to their repositories, hoping they will not Do Something Bad(tm). We have chosen to not provide that trust. > > > > IMO, I do not think the Foundation should relax its stance on provenance, nor trust in third parties ... but that is one of the key considerations [for the Board] at the heart of being able to leverage some third party CI/CD services. > > > > Cheers, > > -g > > > >
