[GitHub] [airflow] BasPH commented on a diff in pull request #29143: Demonstrate usage of the PythonSensor
BasPH commented on code in PR #29143: URL: https://github.com/apache/airflow/pull/29143#discussion_r1091891591 ## docs/apache-airflow/howto/operator/python.rst: ## @@ -225,11 +225,29 @@ Jinja templating can be used in same way as described for the PythonOperator. PythonSensor -Use the :class:`~airflow.sensors.python.PythonSensor` to use arbitrary callable for sensing. The callable -should return True when it succeeds, False otherwise. +A PythonSensor waits for a certain condition to be ``True``, for example to wait for a file to exist. The +PythonSensor is available via ``@task.sensor`` and ``airflow.sensors.python.PythonSensor``. The callable +should return a boolean ``True`` or ``False``, indicating whether a condition is met. For example: -.. exampleinclude:: /../../airflow/example_dags/example_sensors.py -:language: python -:dedent: 4 -:start-after: [START example_python_sensors] -:end-before: [END example_python_sensors] +.. code-block:: python + +import datetime + +from airflow.decorators import dag, task +from airflow.sensors.python import PythonSensor + + +@dag(start_date=datetime.datetime(2023, 1, 1), schedule=None) +def example(): +@task.sensor +def wait_for_success(): +return datetime.datetime.now().minute % 2 == 0 + +wait_for_success() +PythonSensor(task_id="wait_for_even_minute", python_callable=wait_for_success) Review Comment: Separated the functions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] BasPH commented on a diff in pull request #29143: Demonstrate usage of the PythonSensor
BasPH commented on code in PR #29143: URL: https://github.com/apache/airflow/pull/29143#discussion_r1086270155 ## docs/apache-airflow/howto/operator/python.rst: ## @@ -225,11 +225,29 @@ Jinja templating can be used in same way as described for the PythonOperator. PythonSensor -Use the :class:`~airflow.sensors.python.PythonSensor` to use arbitrary callable for sensing. The callable -should return True when it succeeds, False otherwise. +A PythonSensor waits for a certain condition to be ``True``, for example to wait for a file to exist. The +PythonSensor is available via ``@task.sensor`` and ``airflow.sensors.python.PythonSensor``. The callable +should return a boolean ``True`` or ``False``, indicating whether a condition is met. For example: Review Comment: Meh not really, I feel this page intends to give an overview of which operators/sensors exist to orchestrate arbitrary Python functions, but not to show nitty-gritty details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] BasPH commented on a diff in pull request #29143: Demonstrate usage of the PythonSensor
BasPH commented on code in PR #29143: URL: https://github.com/apache/airflow/pull/29143#discussion_r1086270155 ## docs/apache-airflow/howto/operator/python.rst: ## @@ -225,11 +225,29 @@ Jinja templating can be used in same way as described for the PythonOperator. PythonSensor -Use the :class:`~airflow.sensors.python.PythonSensor` to use arbitrary callable for sensing. The callable -should return True when it succeeds, False otherwise. +A PythonSensor waits for a certain condition to be ``True``, for example to wait for a file to exist. The +PythonSensor is available via ``@task.sensor`` and ``airflow.sensors.python.PythonSensor``. The callable +should return a boolean ``True`` or ``False``, indicating whether a condition is met. For example: Review Comment: Meh not really, I feel this page exists to give an overview of which operators/sensors exist to orchestrate arbitrary Python functions, but not to show nitty-gritty details. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [airflow] BasPH commented on a diff in pull request #29143: Demonstrate usage of the PythonSensor
BasPH commented on code in PR #29143: URL: https://github.com/apache/airflow/pull/29143#discussion_r1086267570 ## docs/apache-airflow/howto/operator/python.rst: ## @@ -225,11 +225,29 @@ Jinja templating can be used in same way as described for the PythonOperator. PythonSensor -Use the :class:`~airflow.sensors.python.PythonSensor` to use arbitrary callable for sensing. The callable -should return True when it succeeds, False otherwise. +A PythonSensor waits for a certain condition to be ``True``, for example to wait for a file to exist. The +PythonSensor is available via ``@task.sensor`` and ``airflow.sensors.python.PythonSensor``. The callable +should return a boolean ``True`` or ``False``, indicating whether a condition is met. For example: -.. exampleinclude:: /../../airflow/example_dags/example_sensors.py -:language: python -:dedent: 4 -:start-after: [START example_python_sensors] -:end-before: [END example_python_sensors] +.. code-block:: python + +import datetime + +from airflow.decorators import dag, task +from airflow.sensors.python import PythonSensor + + +@dag(start_date=datetime.datetime(2023, 1, 1), schedule=None) +def example(): +@task.sensor +def wait_for_success(): +return datetime.datetime.now().minute % 2 == 0 + +wait_for_success() +PythonSensor(task_id="wait_for_even_minute", python_callable=wait_for_success) Review Comment: I tested on 2.5.1 and it just works: ![image](https://user-images.githubusercontent.com/6249654/214501338-60dc8b45-7989-48fe-8881-16baf0344de5.png) I could split this into separate functions because it does look a little odd to reference a taskflow-decorated function from a PythonSensor class? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org