ryanyuan commented on a change in pull request #5566: [AIRFLOW-4935] Add method
in the bigquery hook to list tables in a dataset
URL: https://github.com/apache/airflow/pull/5566#discussion_r302793679
##########
File path: airflow/contrib/hooks/bigquery_hook.py
##########
@@ -1718,6 +1718,79 @@ def get_datasets_list(self, project_id=None):
return datasets_list
+ def get_dataset_tables_list(self,dataset_id, project_id=None,
table_prefix=None, max_results=None):
+ """
+ Method returns tables list of a BigQuery dataset. If table prefix is
specified, only tables beginning by it are
+ returned.
+
+ .. seealso::
+ For more information, see:
+
https://cloud.google.com/bigquery/docs/reference/rest/v2/tables/list
+
+ :param dataset_id: The BigQuery Dataset ID
+ :type dataset_id: str
+ :param project_id: The GCP Project ID
+ :type project_id: str
+ :param table_prefix: Tables must begin by this prefix to be returned
+ :type table_prefix: str
+ :param max_results:
+ :type max_results: int
+ :return: dataset_tables_list
+ :type: list(tableReference)
+
+ Example of returned datasets_list:
+
+ [
+ {
+ "projectId": "project1",
+ "datasetId": "dataset1",
+ "tableId": "table1"
+ },
+ {
+ "projectId": "project2",
+ "datasetId": "dataset2",
+ "tableId": "table2"
+ }
+ ]
+ """
+ dataset_project_id = project_id if project_id else self.project_id
+
+ optional_params = {}
+ if max_results:
+ optional_params['maxResults'] = max_results
+
+ dataset_tables_list = []
+ next_page_token = None
+ while True:
+ if next_page_token:
Review comment:
This if check can be gone since next_page_token will be checked later:
`
if not next_page_token:
`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
