shahar1 commented on code in PR #66962:
URL: https://github.com/apache/airflow/pull/66962#discussion_r3243619729
##########
providers/google/src/airflow/providers/google/cloud/sensors/bigquery.py:
##########
@@ -332,6 +333,14 @@ class
BigQueryStreamingBufferEmptySensor(BaseSensorOperator):
``UPDATE/MERGE/DELETE statement over table ... would affect rows in the
streaming buffer`` errors.
+ .. warning::
+ The sensor reads ``table.streaming_buffer`` from BigQuery's table
+ metadata, which is eventually consistent. For a short window right
+ after a streaming insert the buffer metadata is still absent, so the
+ sensor may report the buffer empty before it actually is. Known
+ limitation tracked at
+ https://github.com/apache/airflow/issues/66963
Review Comment:
Flaw in the existing design, can be fixed later
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
