[
https://issues.apache.org/jira/browse/AIRFLOW-2772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Iuliia Volkova reassigned AIRFLOW-2772:
---------------------------------------
Assignee: Iuliia Volkova
> BigQuery hook does not allow specifying both the partition field name and
> table name at the same time
> -----------------------------------------------------------------------------------------------------
>
> Key: AIRFLOW-2772
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2772
> Project: Apache Airflow
> Issue Type: Bug
> Components: hooks
> Reporter: Berislav Lopac
> Assignee: Iuliia Volkova
> Priority: Minor
>
> When creating a load job for a single partition in a BigQuery's partitioned
> table, it is possible to specify either the table name with the partition
> (e.g. {{dataset_name.table_name$partition_id}}), or the field used for the
> partition (e.g. {{time_partitioning=\{"field": "field_name"\}}}) -- but not
> both.
> This is the code that raises the exception, at the very end of
> {{contrib/hooks/bigquery_hook.py}}:
> {code}
> assert not time_partitioning_in.get('field'), (
> "Cannot specify field partition and partition name "
> "(dataset.table$partition) at the same time"
> )
> {code}
> My first problem is using {{assert}} for flow control, but more importantly
> it is not clear what is the rationale for this check and the error if both
> are defined? The code works well if we provide just the partition field
> specification, but passing only the partition table name results in the
> following BQ error:
> {code}Incompatible table partitioning specification. Expects partitioning
> specification interval(type:day,field:local_event_start_date), but input
> partitioning specification is interval(type:day){code}
> which implies that sending both should be perfectly fine.
> Can anyone provide any insight?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
