nathadfield opened a new issue, #34010:
URL: https://github.com/apache/airflow/issues/34010

   ### Apache Airflow version
   
   2.7.0
   
   ### What happened
   
   When running `BigQueryValueCheckOperator` in deferrable mode, the operator 
always reports a successful status even if the `pass_value` has not been met. 
   
   ### What you think should happen instead
   
   If the value returned by the SQL given to the operator does not equal the 
`pass_value` then the operator should fail.  This occurs when 
`deferrable=False` but not when it is `True`.
   
   ### How to reproduce
   
   The following DAG code should replicate the issue.  Both tasks provide some 
SQL that just returns `false` and with a `pass_value` of `True`.  The only 
difference is the fact that the first task is running in deferrable mode.
   
   ```
   from datetime import datetime
   
   from airflow import models
   from airflow.providers.google.cloud.operators.bigquery import 
BigQueryValueCheckOperator
   
   with models.DAG(
       dag_id='bq_value_check',
       start_date=datetime(2023, 8, 31),
       catchup=False,
       schedule='0 0 * * *',
   ) as dag:
   
       test1 = BigQueryValueCheckOperator(
           task_id=f'test1',
           sql=f'SELECT false;',
           pass_value=True,
           retries=0,
           deferrable=False,
       )
   
       test2 = BigQueryValueCheckOperator(
           task_id=f'test2',
           sql=f'SELECT false;',
           pass_value=True,
           retries=0,
           deferrable=True,
       )
   ```
   
   <img width="275" alt="Screenshot 2023-09-01 at 14 42 28" 
src="https://github.com/apache/airflow/assets/967119/8ac7539e-dee0-45d6-b1da-b5a9f32986b4";>
   
   
   Some log extracts:
   **test1**
   
   ```
   [2023-09-01, 13:34:53 UTC] {bigquery.py:1596} INFO - Inserting job 
airflow_1693575293576477_43573f106bd19562ebd18f0679e80536
   [2023-09-01, 13:34:54 UTC] {taskinstance.py:1824} ERROR - Task failed with 
exception
   Traceback (most recent call last):
     File 
"/usr/local/lib/python3.10/site-packages/airflow/providers/google/cloud/operators/bigquery.py",
 line 425, in execute
       super().execute(context=context)
     File 
"/usr/local/lib/python3.10/site-packages/airflow/providers/common/sql/operators/sql.py",
 line 857, in execute
       self._raise_exception(error_msg)
     File 
"/usr/local/lib/python3.10/site-packages/airflow/providers/common/sql/operators/sql.py",
 line 187, in _raise_exception
       raise AirflowException(exception_string)
   airflow.exceptions.AirflowException: Test failed.
   Pass value:True
   Tolerance:None
   Query:
   SELECT false;
   Results:
   [False]
   ```
   
   **test2**
   ```
   [2023-09-01, 13:34:53 UTC] {bigquery.py:1596} INFO - Inserting job 
airflow_1693575293256612_92474edc414865ab2efb14bd8b18e24d
   [2023-09-01, 13:34:53 UTC] {bigquery.py:446} INFO - Current state of job 
airflow_1693575293256612_92474edc414865ab2efb14bd8b18e24d is DONE
   [2023-09-01, 13:34:53 UTC] {taskinstance.py:1345} INFO - Marking task as 
SUCCESS. dag_id=bq_value_check, task_id=test2, execution_date=20230831T000000, 
start_date=20230901T133452, end_date=20230901T133453
   [2023-09-01, 13:34:53 UTC] {local_task_job_runner.py:225} INFO - Task exited 
with return code 0
   ```
   
   ### Operating System
   
   n/a
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-google==10.7.0
   
   ### Deployment
   
   Astronomer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to