hi Jorge -- as far as I know, all that's necessary to use BK is the pipeline configuration in the repo and to attach workers to Arrow's (free) account (which I'm happy to add you to). We can likely get a lot of free cloud credits from Google, Microsoft, or Amazon if we wanted to go that route to keep costs down (I could help pay for the cloud bill, too).
The one thing to be aware of with BK is the possible security risks from builds running on pull requests. With reasonable security practices (not putting sensitive data on the build hosts) this should be too much of a problem. On Tue, Jan 5, 2021 at 11:46 AM Jorge Cardoso Leitão <jorgecarlei...@gmail.com> wrote: > > Krisztián, > > I agree with you that there is an ongoing problem with the queue. Thanks > for the tips wrt to WIP. > > Wes, I would be up for moving Rust workflows to Buildkite. Is the > integration in place wrt to reporting to triggers and reporting back to > gihtub? I.e. can we just place a `pipeline.yml` on the repo, or is there > any PMC-specific activity that is blocking us from working on this? > > Best, > Jorge > > > > > On Tue, Jan 5, 2021 at 5:10 PM Wes McKinney <wesmck...@gmail.com> wrote: > > > At the risk of sounding like a broken record -- we are almost > > certainly going to have to move our builds to dedicated infrastructure > > that this community has complete agency over sometime between now and > > 2025. Maybe it will be this year, maybe next year, but to me it is an > > inevitability. I spent quite a bit of time getting Apache Infra to > > enable Buildkite for apache/* repositories precisely so that we have a > > coherent path. I hope to be able to have folks on my team focus on > > this at some point and contribute it to the community once we have the > > bandwidth. > > > > On Tue, Jan 5, 2021 at 10:01 AM Krisztián Szűcs > > <szucs.kriszt...@gmail.com> wrote: > > > > > > On Tue, Jan 5, 2021 at 4:40 PM Jeroen Ooms <jeroeno...@gmail.com> wrote: > > > > > > > > On Tue, Jan 5, 2021 at 1:34 PM Krisztián Szűcs > > > > <szucs.kriszt...@gmail.com> wrote: > > > > > > > > > > Hi, > > > > > > > > > > I'm concerned about the overall feedback time we have on pull > > requests. > > > > > I have a simple PR to make the comment bot working again, but no > > > > > builds are running even after 30 minutes. > > > > > > > > FWIW, GitHub seems to queue jobs by pusher, not by repo. I have the > > > > same problem sometimes with pull requests taking long to build, but > > > > this usually happens when there are large jobs running under my name > > > > in completely unrelated projects. Also note that if you use e.g. > > > > scheduled (cron) actions, then the jobs are executed in the queue of > > > > the github user who last touched the yml file for that workflow. > > > Based on the "Actions" tab the queueing seems chronological to me, > > > although I can imagine the behaviour you described. > > > > > > Has anyone experienced similar clogging on apache/arrow's gha queue > > lately? > >
