shahar1 opened a new pull request, #67009: URL: https://github.com/apache/airflow/pull/67009
The streaming-buffer test path added in #66652 (and made deterministic in #66962) gates `stream_update` / `stream_delete` behind `BigQueryStreamingBufferEmptySensor`, which against real BigQuery can wait up to ~90 minutes for the streaming buffer to flush. That made `bigquery_sensors` too slow for the Google CI maintainers to run on every cycle. This PR: - Reverts `providers/google/tests/system/google/cloud/bigquery/example_bigquery_sensors.py` to its pre-#66652 form (no streaming-buffer / DML chain). - Extracts the streaming-buffer flow — streaming insert with metadata-lag workaround, sync sensor + UPDATE, deferred sensor + DELETE — into a new `example_bigquery_streaming_buffer_sensor.py`, gated behind `RUN_MANUAL_GOOGLE_SYSTEM_TESTS`, the same flag used by `example_dataproc_cancel_on_kill.py`. - Repoints the `BigQueryStreamingBufferEmptySensor` `exampleinclude` blocks in `providers/google/docs/operators/cloud/bigquery.rst` at the new manual file. The sensor / trigger code, unit tests, and `BigQueryHook.insert_all` docstring fix from #66652 / #66962 are untouched. related: #66652, #66962 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Claude Code (Opus 4.7) Generated-by: Claude Code (Opus 4.7) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
