potiuk commented on code in PR #54258: URL: https://github.com/apache/airflow/pull/54258#discussion_r2278366255
########## contributing-docs/08_static_code_checks.rst: ########## @@ -19,491 +19,191 @@ Static code checks ================== The static code checks in Airflow are used to verify that the code meets certain quality standards. -All the static code checks can be run through pre-commit hooks. +All the static code checks can be run through prek hooks. -The pre-commit hooks perform all the necessary installation when you run them -for the first time. See the table below to identify which pre-commit checks require the Breeze Docker images. +The prek hooks perform all the necessary installation when you run them +for the first time. See the table below to identify which prek checks require the Breeze Docker images. You can also run the checks via `Breeze <../dev/breeze/doc/README.rst>`_ environment. **The outline for this document in GitHub is available at top-right corner button (with 3-dots and 3 lines).** -Pre-commit hooks ----------------- +Prek hooks +---------- Pre-commit hooks help speed up your local development cycle and place less burden on the CI infrastructure. -Consider installing the pre-commit hooks as a necessary prerequisite. +Consider installing the prek hooks as a necessary prerequisite. -The pre-commit hooks by default only check the files you are currently working on and make -them fast. Yet, these checks use exactly the same environment as the CI tests +The hooks by default only check the files you are currently working on (and are staged) which makes the +checks rather fast. Yet, these checks use exactly the same environment as the CI tests use. So, you can be sure your modifications will also work for CI if they pass -pre-commit hooks. +prek hooks. -We have integrated the fantastic `pre-commit <https://pre-commit.com>`__ framework -in our development workflow. To install and use it, you need at least Python 3.10 locally. +We have integrated the `prek <https://github.com/j178/prek>`__ framework +in our development workflow. It can be installed in various ways and does not even need ``pip`` or -Installing pre-commit hooks ---------------------------- +``python`` to be installed. It is a drop-in replacement for the legacy ``pre-commit`` tool, but it is +much faster and more feature-rich. It is written in Rust and it is designed to install environments in parallel, +so it is much faster than the ``pre-commit`` tool. + +Installing prek hooks +--------------------- -It is the best to use pre-commit hooks when you have your local virtualenv for -Airflow activated since then pre-commit hooks and other dependencies are -automatically installed. You can also install the pre-commit hooks manually using ``uv`` or ``pip``. +It is the best to use prek hooks when you have your local virtualenv for +Airflow activated since then prek hooks and other dependencies are +automatically installed. You can also install the prek hooks manually using ``uv`` or ``pipx``. .. code-block:: bash - uv tool install pre-commit + uv tool install prek .. code-block:: bash - pip install pre-commit + pipx install prek + +Since we have a lot of hooks and sometimes you want to run them individually, it's advised to install +auto-completion for the ``prek`` command. You can do it by adding the following line to your +``.bashrc`` or ``.zshrc`` file: + +For bash: + +.. code-block:: bash + + eval "$(COMPLETE=bash prek)" # for bash + +For zsh: + +.. code-block:: zsh -After installation, pre-commit hooks are run automatically when you commit the code and they will + eval "$(COMPLETE=zsh prek)" + +Similarly for other shells like fish, powershell, etc. + +After installation, prek hooks are run automatically when you commit the code and they will only run on the files that you change during your commit, so they are usually pretty fast and do -not slow down your iteration speed on your changes. There are also ways to disable the ``pre-commits`` +not slow down your iteration speed on your changes. There are also ways to disable the prek hooks temporarily when you commit your code with ``--no-verify`` switch or skip certain checks that you find -to much disturbing your local workflow. See `Available pre-commit checks <#available-pre-commit-checks>`_ -and `Using pre-commit <#using-pre-commit>`_ +to much disturbing your local workflow. See `Using prek <#using-prek>`_ -The pre-commit hooks use several external linters that need to be installed before pre-commit is run. +The ``prek`` hooks use several external linters that need to be installed before prek is run. Each of the checks installs its own environment, so you do not need to install those, but there are some checks that require locally installed binaries. On Linux, you typically install them with ``sudo apt install``, on macOS - with ``brew install``. -The current list of prerequisites is limited to ``xmllint``: +The current list of prerequisites is limited to ``xmllint`` and ``golang`` if you want to modify +the Golang code.: -- on Linux, install via ``sudo apt install libxml2-utils`` -- on macOS, install via ``brew install libxml2`` +- on Linux, install via ``sudo apt install libxml2-utils golang`` +- on macOS, install via ``brew install libxml2 golang`` -Some pre-commit hooks also require the Docker Engine to be configured as the static -checks are executed in the Docker environment (See table in the -`Available pre-commit checks <#available-pre-commit-checks>`_ . You should build the images -locally before installing pre-commit checks as described in `Breeze docs <../dev/breeze/doc/README.rst>`__. +Some prek hooks also require the Docker Engine to be configured as the static +checks are executed in the Docker environment. You should build the images +locally before installing prek checks as described in `Breeze docs <../dev/breeze/doc/README.rst>`__. Sometimes your image is outdated and needs to be rebuilt because some dependencies have been changed. -In such cases, the Docker-based pre-commit will inform you that you should rebuild the image. +In such cases, the Docker-based prek will inform you that you should rebuild the image. -In case you do not have your local images built, the pre-commit hooks fail and provide -instructions on what needs to be done. +Enabling prek hooks +------------------- -Enabling pre-commit hooks -------------------------- - -To turn on pre-commit checks for ``commit`` operations in git, enter: +To turn on prek checks for ``commit`` operations in git, enter: .. code-block:: bash - pre-commit install - + prek install To install the checks also for ``pre-push`` operations, enter: .. code-block:: bash - pre-commit install -t pre-push - + prek install -t pre-push For details on advanced usage of the install method, use: .. code-block:: bash - pre-commit install --help + prek install --help -Available pre-commit checks ---------------------------- +.. note:: + + The ``prek`` tool is a drop-in replacement for the legacy ``pre-commit`` tool - much faster and more + feature-rich, If you have already installed ``pre-commit`` to handle your hooks, you can run + ``prek install -f`` to replace the existing ``pre-commit`` hooks with the ``prek`` hooks. + +Using prek +---------- -This table lists pre-commit hooks used by Airflow. The ``Image`` column indicates which hooks Review Comment: Yeah. It's a bit paint to keep it updated - always some conflicts to resolve, and since we have `prek list --verbose` we can very easily see the list of available pre-commits with description, and auto-complete has all those descriptions, I think it makes very little sense to keep the table here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
