arsenelupin14 opened a new pull request, #67401: URL: https://github.com/apache/airflow/pull/67401
Closes #66963 BigQuery's streamingBuffer metadata is eventually consistent, causing false negatives in BigQueryStreamingBufferEmptySensor immediately after a streaming insert. This PR introduces a consecutive_empty_checks parameter (by default: 1) to the sensor and its deferrable trigger. By setting this to >1, the sensor will require multiple consecutive empty observations before passing, safely bypassing the lag window. Also removes the manual time.sleep workaround from example_bigquery_streaming_buffer_sensor.py . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
