Re: Add /ready-to-test bot to airflow CI

2020-10-29 Thread Jarek Potiuk
The change is merged. Please rebase your PRs to the latest master and try it. Your PR builds in most of the cases should be much more "gentle" with the other builds. They should only run one default matrix configuration, but then in case of "core" changes, it will have to be followed up by a "ful

Re: Add /ready-to-test bot to airflow CI

2020-10-28 Thread Jarek Potiuk
WOHO! Seems we are ready with review for this finally. That required workarounds for some bugs in GitHub Actions and releasing of my https://github.com/potiuk/get-workflow-origin action and Tobiasz's https://github.com/TobKed/label-when-approved-action/ more than once (more than several times ;) bu

Re: Add /ready-to-test bot to airflow CI

2020-10-28 Thread Jarek Potiuk
We found a solution, I think, to permission problems and proceed with that. However, we also had a discussion with Tomek and Tobiasz and we think that we can slightly change the messages and "process" to further optimize the number of jobs: We think that there is a really small number of tests t

Re: Add /ready-to-test bot to airflow CI

2020-10-27 Thread Jarek Potiuk
The PR with this "limited tests" PRs is ready for review https://github.com/apache/airflow/pull/11828 Not sure how much it's going to help before we have self-hosted runners, but it's worth trying. J. On Tue, Oct 27, 2020 at 8:51 AM Jarek Potiuk wrote: > BTW. the Action from Tobiasz is alread

Re: Add /ready-to-test bot to airflow CI

2020-10-27 Thread Jarek Potiuk
BTW. the Action from Tobiasz is already out there :) - he just adds the comment/check option now: https://github.com/TobKed/label-when-approved-action/ :D On Tue, Oct 27, 2020 at 8:27 AM Ash Berlin-Taylor wrote: > I think having a test/check status they shows in progress until approved > is actu

Re: Add /ready-to-test bot to airflow CI

2020-10-27 Thread Ash Berlin-Taylor
I think having a test/check status they shows in progress until approved is actually a good thing - it makes it more explicit that there are more tests to come. On 27 October 2020 07:22:00 GMT, Jarek Potiuk wrote: >We are close to the finish, but we've hit some GH limitations with >Tobiasz. >It

Re: Add /ready-to-test bot to airflow CI

2020-10-27 Thread Jarek Potiuk
We are close to the finish, but we've hit some GH limitations with Tobiasz. It turned out that the re-run workflow API ( https://docs.github.com/en/free-pro-team@latest/rest/reference/actions#re-run-a-workflow) has an undocumented feature :) - it only allows to re-run failed runs, but it does not w

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Kaxil Naik
I am happy with "okay to test" / "run tests" . On Mon, Oct 26, 2020, 10:13 Jarek Potiuk wrote: > Kamil - "Ready for review" is not good - it must have been reviewed > already because it has at least one approval. > > Ash - I am ok with "okay to test" :). Hard to mistake it with > anything else

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Jarek Potiuk
Kamil - "Ready for review" is not good - it must have been reviewed already because it has at least one approval. Ash - I am ok with "okay to test" :). Hard to mistake it with anything else and serves the purpose well :) Any other opinions/voices :)? I already have the PRs to enable it in review,

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Ash Berlin-Taylor
How about "okay to test" -- that's often the "command" that people use for test approval (thinking of Jenkins Github integration, where you can say "ready to test" to do this exact purpose). -ash On Oct 26 2020, at 10:06 am, Kamil Breguła wrote: > what do you think about "Ready for review"? > >

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Kamil Breguła
what do you think about "Ready for review"? On Mon, Oct 26, 2020, 11:04 Jarek Potiuk wrote: > On Mon, Oct 26, 2020 at 10:53 AM Ash Berlin-Taylor wrote: > >> Is "ready to merge" also going to automatically merge if tests are green? >> > > Not at all. It was never the intention. Committers still

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Jarek Potiuk
On Mon, Oct 26, 2020 at 10:53 AM Ash Berlin-Taylor wrote: > Is "ready to merge" also going to automatically merge if tests are green? > Not at all. It was never the intention. Committers still need to merge it manually. The difference is that you will see the "Ready to Marge" label and "green" (

Re: Add /ready-to-test bot to airflow CI

2020-10-26 Thread Ash Berlin-Taylor
Is "ready to merge" also going to automatically merge if tests are green? I think it shouldn't unless we also remove that label on a new push to the branch - consider this: PR is reviewed and approved and a simple change, committer reviews it and gives it an approval; tests currently running Lab

Re: Add /ready-to-test bot to airflow CI

2020-10-24 Thread Daniel Imberman
I think ready to merge makes more sense via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2] On Sat, Oct 24, 2020 at 12:13 PM, Jarek Potiuk wrote: Some short update - seems like we can get 50% 60% saving in job usage by the "unapproved PRs". We

Re: Add /ready-to-test bot to airflow CI

2020-10-24 Thread Jarek Potiuk
Some short update - seems like we can get 50% 60% saving in job usage by the "unapproved PRs". We are progressing with implementation :D. On Sat, Oct 24, 2020 at 10:55 AM Jarek Potiuk wrote: > FYI. We found out with Tobiasz, that it will be a bit better and if we add > "*Approved*" label in the

Re: Add /ready-to-test bot to airflow CI

2020-10-24 Thread Jarek Potiuk
FYI. We found out with Tobiasz, that it will be a bit better and if we add " *Approved*" label in the PR that the workflow will automatically set when the issue gets approved. This way we have state of the PR approval (we know when it changes) and know that we should re-run last "small matrix" suc

Re: Add /ready-to-test bot to airflow CI

2020-10-23 Thread Daniel Imberman
+1000! via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.51&pv=10.15.6&source=email_footer_2] On Fri, Oct 23, 2020 at 8:22 AM, Jarek Potiuk wrote: Thanks Tobiasz :). fantastic. I prepared a very short and simple design doc https://docs.google.com/document/d/16rwyCfyDpKWN-DrLYbhj

Re: Add /ready-to-test bot to airflow CI

2020-10-23 Thread Jarek Potiuk
Thanks Tobiasz :). fantastic. I prepared a very short and simple design doc https://docs.google.com/document/d/16rwyCfyDpKWN-DrLYbhjU0B1D58T1RFYan5ltmw4DQg/edit# where we can collaborate. I also added you as collaborator to https://github.com/potiuk/get-workflow-origin that we already use, and I

Re: Add /ready-to-test bot to airflow CI

2020-10-22 Thread Tobiasz Kędzierski
Hi Jarek. sounds good to me. I am happy to help you as much as I can with it. BR Tobiasz On Thu, Oct 22, 2020 at 9:06 AM Jarek Potiuk wrote: > *TLDR; I thought about it a bit and I have a proposal on how to solve it > even better - one that can be implemented over the weekend (I volunteer :) >

Re: Add /ready-to-test bot to airflow CI

2020-10-22 Thread Jarek Potiuk
*TLDR; I thought about it a bit and I have a proposal on how to solve it even better - one that can be implemented over the weekend (I volunteer :) ) and that can be very easily shared and adopted by the other ASF projects so that we all collectively decrease the strain on Github Actions. * This i

Re: Add /ready-to-test bot to airflow CI

2020-10-02 Thread Jarek Potiuk
I think it's a good idea, but I'd augment it a bit. A better option will be to run all test types but for only one chosen combination of "Python + DB type + DB version". I often don't even look at the PR until the tests pass and this would be much better this way. And often people have slower/sm

Re: Add /ready-to-test bot to airflow CI

2020-10-01 Thread Daniel Imberman
I’m not too worried about that. I think people would learn pretty quickly. It hasn’t been an issue for the kubernetes community so I can’t imagine it being an issue for us. End-of-day, we only have a limited amount of compute power and this will increase the speed we merge the PR’s that have pas

Re: Add /ready-to-test bot to airflow CI

2020-10-01 Thread Tomasz Urbaszek
I think I can agree. Especially with flaky tests, some contributors may be confused that some of the tests don't work on CI but work locally... Checking the code quality is good first step. Once there's a review we can start tests on CI. On the other hand, I can see people asking for starting the

Add /ready-to-test bot to airflow CI

2020-10-01 Thread Daniel Imberman
Hello all, With the recent uptick in airflow contribution and pull requests, I have a proposal that I hope will ensure that we do not find ourselves in a CI backlog hell. I noticed that on the Kubernetes project, pull requests do not run integration test until a committer submits a "ready to te