thejens opened a new pull request #10218: URL: https://github.com/apache/airflow/pull/10218
With this change we implement a new sensor that checks the existence of a partition in a partitioned BigQuery table. Currently you can check whether a table exists with the BigQueryTableExistenceSensor, however a common use case is for tasks to append partitions to a table in daily batch executions, or to stream data into hourly/date/int-range partitions. A partition is a common artifact from jobs and systems, and it is a common use case to want to ensure that a partition exists before reading data from it downstream, and with this change you can, using BigQueryTablePartitionExistenceSensor. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
