[
https://issues.apache.org/jira/browse/AIRFLOW-2522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16525600#comment-16525600
]
ASF subversion and git services commented on AIRFLOW-2522:
----------------------------------------------------------
Commit f45a5c8ed47605974775cc89799433f5a793ee3b in incubator-airflow's branch
refs/heads/v1-10-test from Tim Swast
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=f45a5c8 ]
[AIRFLOW-2512][AIRFLOW-2522] Use google-auth instead of oauth2client
* Updates the GCP hooks to use the google-auth
library and removes
dependencies on the deprecated oauth2client
package.
* Removes inconsistent handling of the scope
parameter for different
auth methods.
Note: using google-auth for credentials requires a
newer version of the
google-api-python-client package, so this commit
also updates the
minimum version for that.
To avoid some annoying warnings about the
discovery cache not being
supported, so disable the discovery cache
explicitly as recommend here:
https://stackoverflow.com/a/44518587/101923
Tested by running:
nosetests
tests/contrib/operators/test_dataflow_operator.py
\
tests/contrib/operators/test_gcs*.py \
tests/contrib/operators/test_mlengine_*.py \
tests/contrib/operators/test_pubsub_operator.py \
tests/contrib/hooks/test_gcp*.py \
tests/contrib/hooks/test_gcs_hook.py \
tests/contrib/hooks/test_bigquery_hook.py
and also tested by running some GCP-related DAGs
locally, such as the
Dataproc DAG example at
https://cloud.google.com/composer/docs/quickstart
Closes #3488 from tswast/google-auth
(cherry picked from commit 0f4d681f6f6e15acd1399dede146e75cb688d536)
Signed-off-by: Bolke de Bruin <bo...@xs4all.nl>
> Cannot use GOOGLE_APPLICATION_CREDENTIALS to authenticate for GCP connections
> -----------------------------------------------------------------------------
>
> Key: AIRFLOW-2522
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2522
> Project: Apache Airflow
> Issue Type: Bug
> Components: contrib
> Reporter: Tim Swast
> Assignee: Tim Swast
> Priority: Major
> Fix For: 1.10.0, 2.0.0
>
>
> If you try to use the GOOGLE_APPLICATION_CREDENTIALS environment variable
> with a service account key to authenticate to Google Cloud, as described at
> [https://cloud.google.com/docs/authentication/production] you get an error
> "HttpAccessTokenRefreshError: invalid_scope: Empty or missing scope not
> allowed."
> This error occurs even if you fill in the scope field of the GCP connection.
> The root cause is that scopes are ignored by the GCP hook when using
> application default credentials. They should not be ignored when the default
> credentials are using a service account. (And probably shouldn't be ignored
> at all, preferring an error when scopes are filled in but don't apply to the
> credential type)
> I'll try to fix this while I'm working on
> https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-2512.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
