Thanks Jarek, thats really very helpful. i have enabled it now :) Pavan
On Wed, 22 Oct 2025 at 07:18, Jarek Potiuk <[email protected]> wrote: > > That feels like a personal attack haha! > > Well, I have a very complex love <-> hate relationship with MyPy, and it > has its ups and downs. And ... well I was among the many that often > cancelled mypy without waiting - this is one of the reasons it was > generally put at the end of the .pre-commit-config.yaml... Sf attack - it > was self inflicting :D > > J. > > > On Wed, Oct 22, 2025 at 7:24 AM Amogh Desai <[email protected]> wrote: > > > Thanks Jarek. That looks like a nice improvement for sure. > > > > > and I guess many people just cancelled > > > > That feels like a personal attack haha! > > > > Thanks & Regards, > > Amogh Desai > > > > > > On Tue, Oct 21, 2025 at 1:14 PM Jarek Potiuk <[email protected]> wrote: > > > > > Also another - related QOL improvement/suggestion. I noticed that often > > > when I push my change to my fork, I have many more files detected with > > > `pre-push` enabled. This is because often `main` of my fork is not > synced > > > with the main of Airflow. > > > > > > The way how I solved that I created a small alias (with the help of > `gh`) > > > to sync my fork (origin is the one I use for my private fork) > > > > > > ``` > > > alias sync-repo='gh repo sync potiuk/airflow --branch main && git fetch > > > origin' > > > ``` > > > > > > On Sun, Oct 19, 2025 at 2:43 AM Jarek Potiuk <[email protected]> wrote: > > > > > > > Replace airflow mypy with one of those of course :) > > > > > > > > > > > > On Sun, Oct 19, 2025 at 2:42 AM Jarek Potiuk <[email protected]> > wrote: > > > > > > > >> Oh absolutely. At last PyCon I met the Pyrefly team and we even ran > it > > > >> together on Airflow, and of course we know the team at Astral and > ty. > > > The > > > >> issue at that time was that neither was close to handle Airflow > > (Pyrefly > > > >> out of the box was >10 thousands issues) and ty was very early (stil > > is) > > > >> but mid-term goal is to replace airflow with one of those. > > > >> > > > >> One of the problems we have is that some of our type-checking > depends > > on > > > >> custom mypy plugins and none of those has any support for it (did > not > > > check > > > >> zuban -but it seems one-man show, which is a bit worrying for such a > > > >> complex thing like Python typechecker). > > > >> > > > >> But yeah. If someone would like to give it a shot and see how far we > > are > > > >> and maybe lead the effort of switching to one of those... Absolutely > > :) > > > >> > > > >> J. > > > >> > > > >> On Sun, Oct 19, 2025 at 2:34 AM Dev iL <[email protected]> wrote: > > > >> > > > >>> Thanks Jarek, I know this will definitely make a big difference for > > me! > > > >>> > > > >>> It might be a good opportunity to recommend to those of us who want > > > >>> dev-time type checking to look into Astral's ty ( > > > >>> https://github.com/astral-sh/ty), Meta's pyrefly ( > > > >>> https://github.com/facebook/pyrefly), or David Halter's zuban ( > > > >>> https://github.com/zubanls/zuban/) - all of which are written in > > Rust > > > >>> and > > > >>> are much faster than mypy. > > > >>> > > > >>> On Sat, 18 Oct 2025, 23:13 Jarek Potiuk, <[email protected]> wrote: > > > >>> > > > >>> > Hello Everyone, > > > >>> > > > > >>> > I just merged something that might turn into quite a QOL > > improvement > > > >>> for > > > >>> > all contributors (it looks like a significant QOL improvement for > > my > > > >>> > workflows at least): > https://github.com/apache/airflow/pull/56829 > > > >>> > > > > >>> > This PR moves `mypy` checks to the "pre-push" stage of prek hoos > > from > > > >>> > "pre-commit". You might want to run `prek install --refresh` now. > > to > > > >>> take > > > >>> > advantage of it. And follow up with `prek install --hook-stage > > > >>> pre-push` to > > > >>> > complete it (opt-in). > > > >>> > > > > >>> > I guess a number of people refrained from `prek install` so far > > > because > > > >>> > `mypy` checks were generally slow and required `breeze ci-image > > build > > > >>> > --python 3.10` to be run not only once but also kept up to date. > > > While > > > >>> we > > > >>> > have also other hooks that require `ci-image` - mypy is one that > > was > > > >>> > triggered by pretty much any python file change - and I guess > many > > > >>> people > > > >>> > just cancelled it or even disabled prek auto-hooks because of the > > > >>> waiting > > > >>> > time at commit time. > > > >>> > > > > >>> > With this change - mypy hooks are moved to `pre-push` stage - > this > > > >>> stage is > > > >>> > not enabled when you run `prek install`, you need to explicitly > > > enable > > > >>> it > > > >>> > by `prek install --hook-type pre-push`. So you might opt-in to it > > (I > > > >>> did) > > > >>> > while not impacting regular pre-commit stage hooks. > > > >>> > > > > >>> > What it really means for contributor's workflows: > > > >>> > > > > >>> > * If you did `prek install`, the `git commit` operation should be > > > >>> faster > > > >>> > now and will not run mypy checks, If you did not, good idea is to > > run > > > >>> it > > > >>> > now. I highly recommend doing so. Lots of things are auto-fixed > > when > > > >>> you do > > > >>> > and you save precious push -> fix -> push cycle. > > > >>> > > > > >>> > * if you did (or will do) `prek install --hook-type pre-push` - > all > > > >>> > pre-commit and mypy hooks will run at the "git push" time - but > > this > > > is > > > >>> > more opt-in now, You can always do `git push --no-verify` as with > > > >>> commit to > > > >>> > skip that step though > > > >>> > > > > >>> > * if you did not run `prek install` at all beforr, but keep on > > > running > > > >>> > `prek` manually, this will also speed up plain `prek` execution. > > The > > > >>> > default for `prek` is to run `pre-comit` hook stage - so if you > run > > > >>> `prek` > > > >>> > locally - it will run all the checks for your staged changes - > but > > > not > > > >>> mypy > > > >>> > any more. You will need to run `prek run --hook-stage pre-push` > to > > > run > > > >>> also > > > >>> > mypy checks or run "manual" (full folder) version of checks `prek > > run > > > >>> > --hook stage manual` (optionally wiht --all-files to force full > > > check). > > > >>> > > > > >>> > I hope this will improve QOL and iteration speed for a number of > > > >>> > contributors. Also If there are any ideas, problems, obstacles, > > > >>> > difficulties that the current setup causes - discussing it in > > > devlist, > > > >>> > #contributors channel in slack is a good idea. Also the CI/DEV > > stuff > > > >>> is not > > > >>> > as hard, and many people already contribute - regularly or > > casually, > > > >>> and I > > > >>> > think all of us in the ci/dev team are happy to get feedback and > > > ideas > > > >>> from > > > >>> > everyone contributing. > > > >>> > > > > >>> > Just a side comment - honestly I have no idea why I had not > thought > > > >>> about > > > >>> > such setup before with `pre-push` hooks for mypy. It seems pretty > > > >>> "obvious" > > > >>> > when you ask now, I think it's partially caused by blind-spot > > > developed > > > >>> > from years of doing stuff "this way". > > > >>> > > > > >>> > So any fresh and out-of-the box ideas are more than welcome and > we > > > >>> have now > > > >>> > a strong team of CI/Dev peoople who will hear it and respond. I > > think > > > >>> in > > > >>> > the CI/DEV team we like to do stuff our ways, but we like even > more > > > if > > > >>> we > > > >>> > can learn new tricks (speaking as an Old Dog). > > > >>> > > > > >>> > J. > > > >>> > > > > >>> > > > >> > > > > > >
