maroshmka opened a new issue #12520: URL: https://github.com/apache/airflow/issues/12520
**Description** Schema path (schema_object) for table in `GCSToBigQueryOperator` is downloading schema from source bucket. New feature should support support also pulling schemas from different bucket. **Use case / motivation** I guess there are lot of use-cases. To mention one concrete: - we want to have schemas at 1 place, for us its terraform, where we define all resources and ACLs - those schemas are exported to GCS by CI - we want to use them in loads to BQ tables In case there is a schema change, we make 1 change (in terraform) and it works. Otherwise we would need to do terraform change, apply, then either update `schema_fields` in operator and release it or to allow adding fields on table load (which we don't necessarily want all the time). **Related Issues** Not directly, but at least - https://github.com/apache/airflow/issues/8280 - polish system tests to operator while at it ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
