[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator
nuclearpinguin commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator URL: https://github.com/apache/airflow/pull/7126#discussion_r371205169 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -1109,6 +1110,16 @@ def run_table_upsert(self, dataset_id: str, table_resource: Dict, """ service = self.get_service() # check to see if the table exists +if 'tableReference' not in table_resource: +raise AirflowBadRequest( +'"tableReference" is required within table_resource parameter. ' +'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource' +) +if 'tableId' not in table_resource["tableReference"]: +raise AirflowBadRequest( +'"tableId" is required within table_resource["tableReference"]. ' +'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource' +) Review comment: I just wonder, does it break backward compatibility? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator
nuclearpinguin commented on a change in pull request #7126: [AIRFLOW-6405] Add GCP BigQuery Table Upsert Operator URL: https://github.com/apache/airflow/pull/7126#discussion_r376326526 ## File path: airflow/gcp/hooks/bigquery.py ## @@ -1109,6 +1110,16 @@ def run_table_upsert(self, dataset_id: str, table_resource: Dict, """ service = self.get_service() # check to see if the table exists +if 'tableReference' not in table_resource: +raise AirflowBadRequest( +'"tableReference" is required within table_resource parameter. ' +'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource' +) +if 'tableId' not in table_resource["tableReference"]: +raise AirflowBadRequest( +'"tableId" is required within table_resource["tableReference"]. ' +'See https://cloud.google.com/bigquery/docs/reference/v2/tables#resource' +) Review comment: Users can use this method in their own custom operators that are not a part of Airflow. In this case, we have to be sure that backward compatibility is preserved... especially because it's a BQ which is quite popular in Airflow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
