Also another - related QOL improvement/suggestion. I noticed that often when I push my change to my fork, I have many more files detected with `pre-push` enabled. This is because often `main` of my fork is not synced with the main of Airflow.
The way how I solved that I created a small alias (with the help of `gh`) to sync my fork (origin is the one I use for my private fork) ``` alias sync-repo='gh repo sync potiuk/airflow --branch main && git fetch origin' ``` On Sun, Oct 19, 2025 at 2:43 AM Jarek Potiuk <[email protected]> wrote: > Replace airflow mypy with one of those of course :) > > > On Sun, Oct 19, 2025 at 2:42 AM Jarek Potiuk <[email protected]> wrote: > >> Oh absolutely. At last PyCon I met the Pyrefly team and we even ran it >> together on Airflow, and of course we know the team at Astral and ty. The >> issue at that time was that neither was close to handle Airflow (Pyrefly >> out of the box was >10 thousands issues) and ty was very early (stil is) >> but mid-term goal is to replace airflow with one of those. >> >> One of the problems we have is that some of our type-checking depends on >> custom mypy plugins and none of those has any support for it (did not check >> zuban -but it seems one-man show, which is a bit worrying for such a >> complex thing like Python typechecker). >> >> But yeah. If someone would like to give it a shot and see how far we are >> and maybe lead the effort of switching to one of those... Absolutely :) >> >> J. >> >> On Sun, Oct 19, 2025 at 2:34 AM Dev iL <[email protected]> wrote: >> >>> Thanks Jarek, I know this will definitely make a big difference for me! >>> >>> It might be a good opportunity to recommend to those of us who want >>> dev-time type checking to look into Astral's ty ( >>> https://github.com/astral-sh/ty), Meta's pyrefly ( >>> https://github.com/facebook/pyrefly), or David Halter's zuban ( >>> https://github.com/zubanls/zuban/) - all of which are written in Rust >>> and >>> are much faster than mypy. >>> >>> On Sat, 18 Oct 2025, 23:13 Jarek Potiuk, <[email protected]> wrote: >>> >>> > Hello Everyone, >>> > >>> > I just merged something that might turn into quite a QOL improvement >>> for >>> > all contributors (it looks like a significant QOL improvement for my >>> > workflows at least): https://github.com/apache/airflow/pull/56829 >>> > >>> > This PR moves `mypy` checks to the "pre-push" stage of prek hoos from >>> > "pre-commit". You might want to run `prek install --refresh` now. to >>> take >>> > advantage of it. And follow up with `prek install --hook-stage >>> pre-push` to >>> > complete it (opt-in). >>> > >>> > I guess a number of people refrained from `prek install` so far because >>> > `mypy` checks were generally slow and required `breeze ci-image build >>> > --python 3.10` to be run not only once but also kept up to date. While >>> we >>> > have also other hooks that require `ci-image` - mypy is one that was >>> > triggered by pretty much any python file change - and I guess many >>> people >>> > just cancelled it or even disabled prek auto-hooks because of the >>> waiting >>> > time at commit time. >>> > >>> > With this change - mypy hooks are moved to `pre-push` stage - this >>> stage is >>> > not enabled when you run `prek install`, you need to explicitly enable >>> it >>> > by `prek install --hook-type pre-push`. So you might opt-in to it (I >>> did) >>> > while not impacting regular pre-commit stage hooks. >>> > >>> > What it really means for contributor's workflows: >>> > >>> > * If you did `prek install`, the `git commit` operation should be >>> faster >>> > now and will not run mypy checks, If you did not, good idea is to run >>> it >>> > now. I highly recommend doing so. Lots of things are auto-fixed when >>> you do >>> > and you save precious push -> fix -> push cycle. >>> > >>> > * if you did (or will do) `prek install --hook-type pre-push` - all >>> > pre-commit and mypy hooks will run at the "git push" time - but this is >>> > more opt-in now, You can always do `git push --no-verify` as with >>> commit to >>> > skip that step though >>> > >>> > * if you did not run `prek install` at all beforr, but keep on running >>> > `prek` manually, this will also speed up plain `prek` execution. The >>> > default for `prek` is to run `pre-comit` hook stage - so if you run >>> `prek` >>> > locally - it will run all the checks for your staged changes - but not >>> mypy >>> > any more. You will need to run `prek run --hook-stage pre-push` to run >>> also >>> > mypy checks or run "manual" (full folder) version of checks `prek run >>> > --hook stage manual` (optionally wiht --all-files to force full check). >>> > >>> > I hope this will improve QOL and iteration speed for a number of >>> > contributors. Also If there are any ideas, problems, obstacles, >>> > difficulties that the current setup causes - discussing it in devlist, >>> > #contributors channel in slack is a good idea. Also the CI/DEV stuff >>> is not >>> > as hard, and many people already contribute - regularly or casually, >>> and I >>> > think all of us in the ci/dev team are happy to get feedback and ideas >>> from >>> > everyone contributing. >>> > >>> > Just a side comment - honestly I have no idea why I had not thought >>> about >>> > such setup before with `pre-push` hooks for mypy. It seems pretty >>> "obvious" >>> > when you ask now, I think it's partially caused by blind-spot developed >>> > from years of doing stuff "this way". >>> > >>> > So any fresh and out-of-the box ideas are more than welcome and we >>> have now >>> > a strong team of CI/Dev peoople who will hear it and respond. I think >>> in >>> > the CI/DEV team we like to do stuff our ways, but we like even more if >>> we >>> > can learn new tricks (speaking as an Old Dog). >>> > >>> > J. >>> > >>> >>
