jithin97 commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP
BigQuery Table Upsert Operator
URL: https://github.com/apache/airflow/pull/7126#discussion_r372935998
##########
File path: airflow/gcp/hooks/bigquery.py
##########
@@ -1109,6 +1110,16 @@ def run_table_upsert(self, dataset_id: str,
table_resource: Dict,
"""
service = self.get_service()
# check to see if the table exists
+ if 'tableReference' not in table_resource:
+ raise AirflowBadRequest(
+ '"tableReference" is required within table_resource parameter.
'
+ 'See
https://cloud.google.com/bigquery/docs/reference/v2/tables#resource'
+ )
+ if 'tableId' not in table_resource["tableReference"]:
+ raise AirflowBadRequest(
+ '"tableId" is required within
table_resource["tableReference"]. '
+ 'See
https://cloud.google.com/bigquery/docs/reference/v2/tables#resource'
+ )
Review comment:
@nuclearpinguin If your concern is about the operators then I think we are
safe because currently none of the operators are using this method. The only
calls are from test cases.
Memo:
```
:~/Desktop/search_work/airflow|master
⇒ git grep run_table_upsert
CHANGELOG.txt:- [AIRFLOW-314] Fix BigQuery cursor run_table_upsert method
CHANGELOG.txt:- Add two methods to bigquery hook's base cursor:
run_table_upsert, which adds a table or updates an existing table; and
run_grant_dataset_view_access, which grants view access to a given dataset for
a given table.
UPDATING.md:| airflow.gcp.hooks.bigquery.BigQueryBaseCursor.run_table_upsert
| airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert
|
airflow/gcp/hooks/bigquery.py: def run_table_upsert(self, dataset_id:
str, table_resource: Dict,
airflow/gcp/hooks/bigquery.py: def run_table_upsert(self, *args,
**kwargs) -> Dict:
airflow/gcp/hooks/bigquery.py: Please use
`airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert`
airflow/gcp/hooks/bigquery.py: "Please use
`airflow.gcp.hooks.bigquery.BigQueryHook.run_table_upsert`",
airflow/gcp/hooks/bigquery.py: return
self.hook.run_table_upsert(*args, **kwargs)
tests/gcp/hooks/test_bigquery.py:
bq_hook.run_table_upsert(dataset_id=DATASET_ID, table_resource=table_resource)
tests/gcp/hooks/test_bigquery.py:
bq_hook.run_table_upsert(dataset_id=DATASET_ID, table_resource=table_resource)
tests/gcp/hooks/test_bigquery.py: ("run_table_upsert",),
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services