potiuk commented on PR #43329: URL: https://github.com/apache/airflow/pull/43329#issuecomment-2445278123
> I think using generic caches from GitHub actions have also a couple of actions for caching. Some of them specifically to yarn, npm etc... I don't think we would fully cascade that to the local environment though since it's limited to using it in GitHub CI. Maybe we can consider caching in CI and locally as separate tasks which using GitHub Actions could be easier to maintain in the future. Yeah. That's something that we could do for regular installing of Yarn/NPM dependencies as part of CI jobs, but that will not work well for installing Yarn/NPM deps during the Dockerfile build. And yes - also we have a few places where we install it for local development as you wrote. But I think it's better to use specific caching in each case: * Dockerfile -> there the "tip" caching is really, really efficient, it works great because single build on the server saves all the rebuilds locally - and we just pull single container layer with installed dependencies - which is WAY faster than pulling half of the internet and 1000 node modules individually. * pre-commit - it installs node automatically without us doing anything (when language: node is used). And there we also easily controle node version `default_language_version: python: python3 node: 22.2.0`. There caching is achieved by caching the whole pre-commit cache. And the nice thing is that pre-commit will use single node environment for all the pre-commit hooks that have same specification. We have quite a few of them - some for old UI with yarn, some for new with pnpm and some for a number of node tools. And here I think it already works well with current caching in CI and locally without any change from our side. I think both of them will be difficult to implement with any of the "standard" actions and are generally "better" - and same in CI and locally. So yeah I think your current approach is right. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
