Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-10-18 Thread Driesprong, Fokko
Awesome Jarek, that was my fear as well. Good to hear that we still run the full test suite.\ Cheers, Fokko Op vr 18 okt. 2019 om 15:47 schreef Daniel Imberman < daniel.imber...@gmail.com>: > Oh then that sounds fine in that case :) > > via Newton Mail [ > https://cloudmagic.com/k/d/mailapp?ct=d

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-10-18 Thread Daniel Imberman
Oh then that sounds fine in that case :) via Newton Mail [https://cloudmagic.com/k/d/mailapp?ct=dx&cv=10.0.32&pv=10.14.5&source=email_footer_2] On Fri, Oct 18, 2019 at 10:46 AM, Jarek Potiuk wrote: It's even more than daily. In the latest version every time after we merge to master full suite of

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-10-18 Thread Jarek Potiuk
It's even more than daily. In the latest version every time after we merge to master full suite of tests will be run - which means that we can immediately see which commit caused the problem. It's only at the PR level the tests will be "smart". On Fri, Oct 18, 2019 at 3:43 PM Daniel Imberman wrot

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-10-18 Thread Daniel Imberman
I’m still somewhat hesitant on this as it could allow regressions to peak through, though as long as we’re still doing the daily build, and with our soon-to-be-created prerelease load testing I think we should be okay. On Fri, Oct 18, 2019 at 10:30 AM, Jarek Potiuk wrote: Seems that our tests g

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-10-18 Thread Jarek Potiuk
Seems that our tests got way smarter now :). I just implemented the "smartness" it with https://github.com/apache/airflow/pull/6321 and trying to workaround Kubernetes problem we have :). The doc changes are short enough that there is no need to optimise those. Doc-test only should execute much mu

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-08-23 Thread James Meickle
GitHub recently introduced the idea of "Draft" PRs: https://github.blog/2019-02-14-introducing-draft-pull-requests/ Could we do something similar either with that system or something else? Run a minimal set until it's marked as "ready for testing", and then run a larger suite. On Fri, Aug 23, 201

Re: [DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-08-23 Thread Kaxil Naik
Maybe your 4th point covers this but there are frequent Doc only changes. In this case we should not run "real-test" but only static checks - mypy, pylint. flake8, doc generation. On Fri, Aug 23, 2019 at 2:39 PM Jarek Potiuk wrote: > Hello everyone, > > On top of moving out from Travis (I will r

[DISCUSS] Smarter test execution for CI (trivial changes without full tests)

2019-08-23 Thread Jarek Potiuk
Hello everyone, On top of moving out from Travis (I will resume working on it next week) I thought about some ways to improve the feedback cycle time we have with CI (super long now). Maybe we should consider being smarter with running tests only when really needed. After introducing pre-commit