totogo opened a new issue #15088:
URL: https://github.com/apache/airflow/issues/15088
After a `TransferConfig` is created successfully by operator
`BigQueryCreateDataTransferOperator`, and also confirmed in GCP console, the
created Resource name is:
```
projects/<project-id>/locations/europe/transferConfigs/<transfer-config-id>
```
Then I use operator `BigQueryDataTransferServiceStartTransferRunsOperator`
to run the transfer, I got this error:
```
Traceback (most recent call last)
File "/usr/local/lib/airflow/airflow/models/taskinstance.py", line 985, in
_run_raw_tas
result = task_copy.execute(context=context
File
"/usr/local/lib/airflow/airflow/providers/google/cloud/operators/bigquery_dts.py",
line 290, in execut
metadata=self.metadata
File
"/usr/local/lib/airflow/airflow/providers/google/common/hooks/base_google.py",
line 425, in inner_wrappe
return func(self, *args, **kwargs
File
"/usr/local/lib/airflow/airflow/providers/google/cloud/hooks/bigquery_dts.py",
line 235, in start_manual_transfer_run
metadata=metadata or ()
File
"/opt/python3.6/lib/python3.6/site-packages/google/cloud/bigquery_datatransfer_v1/services/data_transfer_service/client.py",
line 1110, in start_manual_transfer_run
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,
File
"/opt/python3.6/lib/python3.6/site-packages/google/api_core/gapic_v1/method.py",
line 145, in __call_
return wrapped_func(*args, **kwargs
File
"/opt/python3.6/lib/python3.6/site-packages/google/api_core/grpc_helpers.py",
line 75, in error_remapped_callabl
six.raise_from(exceptions.from_grpc_error(exc), exc
File "<string>", line 3, in raise_fro
google.api_core.exceptions.NotFound: 404 Requested entity was not found
```
So I went to the [Google's API reference
page](https://cloud.google.com/bigquery-transfer/docs/reference/datatransfer/rest/v1/projects.locations.transferConfigs/startManualRuns)
to run some test. When I use this parent parameter
`projects/{projectId}/transferConfigs/{configId}`, it thrown the same error.
But it works when I use
`rojects/{projectId}/locations/{locationId}/transferConfigs/{configId}`
I guess the piece of code that causes this issue is here in the hook, why
does it use `projects/{projectId}/transferConfigs/{configId}` instead of
`rojects/{projectId}/locations/{locationId}/transferConfigs/{configId}`?
https://github.com/apache/airflow/blob/def961512904443db90e0a980c43dc4d8f8328d5/airflow/providers/google/cloud/hooks/bigquery_dts.py#L226-L232
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
