[jira] [Assigned] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia reassigned AIRFLOW-4470:
Assignee: Golokesh Patra
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Assignee: Golokesh Patra
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG, airflow_sso4.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Assigned] (AIRFLOW-6647) Reduce the cluttering of Airflow UI by merging create and check into a single CHECK step.
[ https://issues.apache.org/jira/browse/AIRFLOW-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nidhi Chourasia reassigned AIRFLOW-6647: Assignee: Golokesh Patra (was: Nidhi Chourasia) > Reduce the cluttering of Airflow UI by merging create and check into a single > CHECK step. > - > > Key: AIRFLOW-6647 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6647 > Project: Apache Airflow > Issue Type: Improvement > Components: operators, ui >Affects Versions: 1.10.0 > Environment: Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-1027-aws x86_64) >Reporter: Nidhi Chourasia >Assignee: Golokesh Patra >Priority: Minor > > This is another UI Feature which make the Airflow UI > # Clutter Free > # More readable > # More intuitive > Presently on Airflow, for any job there will be 2 steps – CREATE step and > CHECK step. > CREATE STEP – It only creates the jobs/Runs the Job as per the schedule and > dies of immediately > CHECK STEP – After the create step, this step will keep on tracking the > status of the job ( i.e check if the job is in RUNNING/FAILED/SUCCESSFUL > state) > We presently host close to approx. 40 jobs, before our enhancement there > would have been 80 Steps created in the form of a TREE ( DAG ), but after our > enhancement , the steps CREATE and CHECK have been merged on the UI side to > just one single CHECK step. > -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17199922#comment-17199922
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
[~coopergillan] I think it will be a good contribution to the documentation ,if
you add the values under OAUTH_PROVIDER section in webserver_config.py as many
might not stumble across this to get a solution.
The link for the same is
[https://flask-appbuilder.readthedocs.io/en/latest/security.html.]
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Assignee: Massipssa Kerrache
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG, airflow_sso4.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (AIRFLOW-6647) Reduce the cluttering of Airflow UI by merging create and check into a single CHECK step.
[ https://issues.apache.org/jira/browse/AIRFLOW-6647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nidhi Chourasia reassigned AIRFLOW-6647: Assignee: Nidhi Chourasia > Reduce the cluttering of Airflow UI by merging create and check into a single > CHECK step. > - > > Key: AIRFLOW-6647 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6647 > Project: Apache Airflow > Issue Type: Improvement > Components: operators, ui >Affects Versions: 1.10.0 > Environment: Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-1027-aws x86_64) >Reporter: Nidhi Chourasia >Assignee: Nidhi Chourasia >Priority: Minor > > This is another UI Feature which make the Airflow UI > # Clutter Free > # More readable > # More intuitive > Presently on Airflow, for any job there will be 2 steps – CREATE step and > CHECK step. > CREATE STEP – It only creates the jobs/Runs the Job as per the schedule and > dies of immediately > CHECK STEP – After the create step, this step will keep on tracking the > status of the job ( i.e check if the job is in RUNNING/FAILED/SUCCESSFUL > state) > We presently host close to approx. 40 jobs, before our enhancement there > would have been 80 Steps created in the form of a TREE ( DAG ), but after our > enhancement , the steps CREATE and CHECK have been merged on the UI side to > just one single CHECK step. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6647) Reduce the cluttering of Airflow UI by merging create and check into a single CHECK step.
Nidhi Chourasia created AIRFLOW-6647: Summary: Reduce the cluttering of Airflow UI by merging create and check into a single CHECK step. Key: AIRFLOW-6647 URL: https://issues.apache.org/jira/browse/AIRFLOW-6647 Project: Apache Airflow Issue Type: Improvement Components: operators, ui Affects Versions: 1.10.0 Environment: Ubuntu 18.04.1 LTS (GNU/Linux 4.15.0-1027-aws x86_64) Reporter: Nidhi Chourasia This is another UI Feature which make the Airflow UI # Clutter Free # More readable # More intuitive Presently on Airflow, for any job there will be 2 steps – CREATE step and CHECK step. CREATE STEP – It only creates the jobs/Runs the Job as per the schedule and dies of immediately CHECK STEP – After the create step, this step will keep on tracking the status of the job ( i.e check if the job is in RUNNING/FAILED/SUCCESSFUL state) We presently host close to approx. 40 jobs, before our enhancement there would have been 80 Steps created in the form of a TREE ( DAG ), but after our enhancement , the steps CREATE and CHECK have been merged on the UI side to just one single CHECK step. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985518#comment-16985518
]
Nidhi Chourasia edited comment on AIRFLOW-4470 at 12/1/19 10:06 AM:
Hi [~jackjack10],
I think it is a bug in Flask-Appbuilder package which we are using for github
authentication for role based access.
It seems to pickup the value of 'login' instead of 'github' for the variable
'provider'
Attaching the screenshot for reference.
was (Author: nidhi94_):
I think it is a bug in Flask-Appbuilder package which we are using for github
authentication for role based access.
It seems to pickup the value of 'login' instead of 'github' for the variable
'provider'
Attaching the screenshot for reference.
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG, airflow_sso4.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia updated AIRFLOW-4470:
-
Attachment: airflow_sso4.PNG
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG, airflow_sso4.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985520#comment-16985520
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
!airflow_sso4.PNG!
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG, airflow_sso4.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia updated AIRFLOW-4470:
-
Attachment: airflow_sso3.PNG
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985518#comment-16985518
]
Nidhi Chourasia edited comment on AIRFLOW-4470 at 12/1/19 10:03 AM:
I think it is a bug in Flask-Appbuilder package which we are using for github
authentication for role based access.
It seems to pickup the value of 'login' instead of 'github' for the variable
'provider'
Attaching the screenshot for reference.
was (Author: nidhi94_):
I think it is a bug in Flask-Appbuilder package which we are using for github
authentication for role based access.
It seems to pickup the value of 'login' instead of 'github' for the variable
'provider'
!image-2019-12-01-15-31-22-217.png!
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, airflow_sso3.PNG,
> image-2019-10-30-16-25-14-436.png, image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16985518#comment-16985518
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
I think it is a bug in Flask-Appbuilder package which we are using for github
authentication for role based access.
It seems to pickup the value of 'login' instead of 'github' for the variable
'provider'
!image-2019-12-01-15-31-22-217.png!
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983390#comment-16983390
]
Nidhi Chourasia edited comment on AIRFLOW-4470 at 11/27/19 10:56 AM:
-
https://issues.apache.org/jira/browse/AIRFLOW-2992.
is related to redirect uri for google
but this is for github enterprise .
Also this seems to have resolved as provided by Vince
[https://your-airflow-url/oauth-authorized/github]
but there is no documentation for other values to be specified in
webserver_config.py because of which could not get it working end to end.
was (Author: nidhi94_):
https://issues.apache.org/jira/browse/AIRFLOW-2992.
is related to redirect uri for google
but this is for github enterprise .
Also this seems to have resolved as provided by Vince
https://your-airflow-url/oauth-authorized/github
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16983390#comment-16983390
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
https://issues.apache.org/jira/browse/AIRFLOW-2992.
is related to redirect uri for google
but this is for github enterprise .
Also this seems to have resolved as provided by Vince
https://your-airflow-url/oauth-authorized/github
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5864) RBAC with github_enterprise oauth authentication
[
https://issues.apache.org/jira/browse/AIRFLOW-5864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia updated AIRFLOW-5864:
-
Attachment: rbac_error_page.PNG
> RBAC with github_enterprise oauth authentication
>
>
> Key: AIRFLOW-5864
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5864
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.6
> Environment: Linux
>Reporter: Nidhi Chourasia
>Priority: Blocker
> Attachments: rbac_error_page.PNG
>
>
> For the following configuration in webserver_config.py
> {code:java}
> // code placeholder
> OAUTH_PROVIDERS = [{
>'name':'github',
> 'token_key':'access_token',
> 'icon':'fa-github',
> 'remote_app': {
> 'base_url':'https://github.corporate.domain.com/login',
> # 'request_token_params':{
> # 'scope': 'email profile'
> # },
>
> 'access_token_url':'https://github.corporate.domain.com/login/oauth/access_token',
> #'access_token_url': None,
>
> 'authorize_url':'https://github.corporate.domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '**',
> 'consumer_secret': '',
> }
> }]
> {code}
> is redirecting to the following
> !image-2019-11-07-15-59-21-298.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-5864) RBAC with github_enterprise oauth authentication
Nidhi Chourasia created AIRFLOW-5864:
Summary: RBAC with github_enterprise oauth authentication
Key: AIRFLOW-5864
URL: https://issues.apache.org/jira/browse/AIRFLOW-5864
Project: Apache Airflow
Issue Type: Bug
Components: authentication, webserver
Affects Versions: 1.10.6
Environment: Linux
Reporter: Nidhi Chourasia
For the following configuration in webserver_config.py
{code:java}
// code placeholder
OAUTH_PROVIDERS = [{
'name':'github',
'token_key':'access_token',
'icon':'fa-github',
'remote_app': {
'base_url':'https://github.corporate.domain.com/login',
# 'request_token_params':{
# 'scope': 'email profile'
# },
'access_token_url':'https://github.corporate.domain.com/login/oauth/access_token',
#'access_token_url': None,
'authorize_url':'https://github.corporate.domain.com/login/oauth/authorize',
'request_token_url': None,
'consumer_key': '**',
'consumer_secret': '',
}
}]
{code}
is redirecting to the following
!image-2019-11-07-15-59-21-298.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5271) EmrCreateJobFlowOperator throwing error in airflow 1.10.4 version
[
https://issues.apache.org/jira/browse/AIRFLOW-5271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967310#comment-16967310
]
Nidhi Chourasia commented on AIRFLOW-5271:
--
Yes followed the steps and it is working now.[~ash]
> EmrCreateJobFlowOperator throwing error in airflow 1.10.4 version
> -
>
> Key: AIRFLOW-5271
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5271
> Project: Apache Airflow
> Issue Type: Bug
> Components: operators
>Affects Versions: 1.10.4
> Environment: Operating System details:
> ubuntu@ip-10-0-1-252:~$ cat /etc/os-release
> NAME="Ubuntu"
> VERSION="18.04.1 LTS (Bionic Beaver)"
> ID=ubuntu
> ID_LIKE=debian
> PRETTY_NAME="Ubuntu 18.04.1 LTS"
> VERSION_ID="18.04"
> HOME_URL="https://www.ubuntu.com/;
> SUPPORT_URL="https://help.ubuntu.com/;
> BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/;
> PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy;
> VERSION_CODENAME=bionic
> UBUNTU_CODENAME=bionic
>Reporter: Nidhi Chourasia
>Priority: Blocker
>
> h3. *ERROR LOGS:*
> {{[2019-08-21 05:39:42,970] \{emr_create_job_flow_operator.py:66} INFO -
> Creating JobFlow using aws-conn-id: aws_default, emr-conn-id: emr_default
> [2019-08-21 05:39:42,981] \{logging_mixin.py:95} INFO - [[34m2019-08-21
> 05:39:42,980[0m] \{[34mconnection.py:[0m296} ERROR[0m - Expecting
> property name enclosed in double quotes: line 1 column 2 (char 1)[0m
> [31mTraceback (most recent call last):
> File
> "/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models/connection.py",
> line 294, in extra_dejson
> obj = json.loads(self.extra)
> File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
> return _default_decoder.decode(s)
> File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
> obj, end = self.raw_decode(s, idx=_w(s, 0).end())
> File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
> obj, end = self.scan_once(s, idx)
> ValueError: Expecting property name enclosed in double quotes: line 1 column
> 2 (char 1)[0m
> [2019-08-21 05:39:42,982] \{logging_mixin.py:95} INFO - [[34m2019-08-21
> 05:39:42,981[0m] \{[34mconnection.py:[0m297} ERROR[0m - Failed parsing
> the json for conn_id [1maws_default[0m[0m
> [2019-08-21 05:39:43,054] \{taskinstance.py:1047} ERROR - Parameter
> validation failed:
> Unknown parameter in input: "TerminationProtected", must be one of: Name,
> LogUri, AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps,
> BootstrapActions, SupportedProducts, NewSupportedProducts, Applications,
> Configurations, VisibleToAllUsers, JobFlowRole, ServiceRole, Tags,
> SecurityConfiguration, AutoScalingRole, ScaleDownBehavior, CustomAmiId,
> EbsRootVolumeSize, RepoUpgradeOnBoot, KerberosAttributes
> Unknown parameter in input: "KeepJobFlowAliveWhenNoSteps", must be one of:
> Name, LogUri, AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps,
> BootstrapActions, SupportedProducts, NewSupportedProducts, Applications,
> Configurations, VisibleToAllUsers, JobFlowRole, ServiceRole, Tags,
> SecurityConfiguration, AutoScalingRole, ScaleDownBehavior, CustomAmiId,
> EbsRootVolumeSize, RepoUpgradeOnBoot, KerberosAttributes
> Unknown parameter in input: "Ec2SubnetId", must be one of: Name, LogUri,
> AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps, BootstrapActions,
> SupportedProducts, NewSupportedProducts, Applications, Configurations,
> VisibleToAllUsers, JobFlowRole, ServiceRole, Tags, SecurityConfiguration,
> AutoScalingRole, ScaleDownBehavior, CustomAmiId, EbsRootVolumeSize,
> RepoUpgradeOnBoot, KerberosAttributes
> Unknown parameter in input: "Ec2KeyName", must be one of: Name, LogUri,
> AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps, BootstrapActions,
> SupportedProducts, NewSupportedProducts, Applications, Configurations,
> VisibleToAllUsers, JobFlowRole, ServiceRole, Tags, SecurityConfiguration,
> AutoScalingRole, ScaleDownBehavior, CustomAmiId, EbsRootVolumeSize,
> RepoUpgradeOnBoot, KerberosAttributes}}
> h3. {{*CORRESPONDING DAG CODE:*}}
> {{}}
> {noformat}
> // code placeholder
> airflow_test_json=json.load(open(airflow_home+'/test.json'))
> airflow_asset_analytics_creator = EmrCreateJobFlowOperator(
> task_id='create_asset_analytics_databricks_test',
> job_flow_overrides=airflow_test_json['Job'],
> timeout=10,
> aws_conn_id='aws_default',
> emr_conn_id='emr_default',
> dag=dag
> )
> airflow_asset_analytics_sensor = EmrJobFlowSensor(
> task_id='check_asset_analytics_databricks_stable',
> job_flow_id="{{
> task_instance.xcom_pull('create_asset_analytics_databricks_test',
> key='return_value') }}",
> aws_conn_id='aws_default',
> dag=dag
> )
>
[jira] [Comment Edited] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964175#comment-16964175
]
Nidhi Chourasia edited comment on AIRFLOW-4470 at 10/31/19 4:24 PM:
[~vinceatbluelabs] Appreciate your kindness and prompt response.
Thanks for correcting will reach out in the above link
was (Author: nidhi94_):
[~vinceatbluelabs] Appreciate and kindness and prompt response.
Thanks for correcting will reach out in the above link
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964175#comment-16964175
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
[~vinceatbluelabs] Appreciate and kindness and prompt response.
Thanks for correcting will reach out in the above link
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963698#comment-16963698
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
Am I missing something?
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963697#comment-16963697
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
!airflow_ss0_2.PNG!
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia updated AIRFLOW-4470:
-
Attachment: airflow_ss0_2.PNG
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: airflow_ss0_2.PNG, image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nidhi Chourasia updated AIRFLOW-4470:
-
Attachment: image-2019-10-31-11-47-04-041.png
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
> Attachments: image-2019-10-30-16-25-14-436.png,
> image-2019-10-31-11-47-04-041.png
>
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-4470) RBAC Github Enterprise OAuth provider callback URL?
[
https://issues.apache.org/jira/browse/AIRFLOW-4470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962954#comment-16962954
]
Nidhi Chourasia commented on AIRFLOW-4470:
--
Hi , where do we specify the callback_url in webserver_config.py as mentioned
[~vinceatbluelabs].
> RBAC Github Enterprise OAuth provider callback URL?
> ---
>
> Key: AIRFLOW-4470
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4470
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication, webserver
>Affects Versions: 1.10.2
>Reporter: Geez
>Priority: Blocker
> Labels: usability
>
> Hi all,
> Quick question, when using RBAC with OAuth providers (1.10.2):
> * we are not specifying the {{authenticate}} or {{auth_backend}} in the
> [webserver] section of \{{airflow.cfg}}anymore
> * Instead, we set the OAuth provider config in the flask-appbuilder's
> {{webserver_config.py}}:
> {code:java}
>
> # Adapting Google OAuth example to Github:
> OAUTH_PROVIDERS = [
> {'name':'github', 'icon':'fa-github', 'token_key':'access_token',
> 'remote_app': {
> 'base_url':'https://github.corporate-domain.com/login',
>
> 'access_token_url':'https://github.corporate-domain.com/login/oauth/access_token',
>
> 'authorize_url':'https://github.corporate-domain.com/login/oauth/authorize',
> 'request_token_url': None,
> 'consumer_key': '',
> 'consumer_secret': 'X',
> }
> }
> ]
>
> {code}
> _Question:_
> * so what callback URL do we specify in the app?
> {{http:/webapp/ghe_oauth/callback}} would not work right? (example with
> github entreprise)
> No matter what I specify for the callback url (/ghe_oauth/callback or
> [http://webapp.com|http://webapp.com/]), I get an error message about
> {{redirect_uri}} mismatch:
> {code:java}
> {{error=redirect_uri_mismatch_description=The+redirect_uri+MUST+match+the+registered+callback+URL+for+this+application
> }}{code}
> _Docs ref:_
> Here is how you setup OAuth with Github Entreprise on Airflow _*without*_
> RBAC:
> [https://airflow.apache.org/security.html#github-enterprise-ghe-authentication]
> And here is how you setup OAuth via the {{webserver_config.py}} of
> flask_appbuilder used by airflow _*with*_RBAC:
>
> [https://flask-appbuilder.readthedocs.io/en/latest/security.html#authentication-oauth]
> What's the *callback url* when using RBAC and OAuth with Airflow?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-5271) EmrCreateJobFlowOperator throwing error in airflow 1.10.4 version
Nidhi Chourasia created AIRFLOW-5271:
Summary: EmrCreateJobFlowOperator throwing error in airflow 1.10.4
version
Key: AIRFLOW-5271
URL: https://issues.apache.org/jira/browse/AIRFLOW-5271
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.10.4
Environment: Operating System details:
ubuntu@ip-10-0-1-252:~$ cat /etc/os-release
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.1 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/;
SUPPORT_URL="https://help.ubuntu.com/;
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/;
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy;
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
Reporter: Nidhi Chourasia
Fix For: 1.10.5
h3. *ERROR LOGS:*
{{[2019-08-21 05:39:42,970] \{emr_create_job_flow_operator.py:66} INFO -
Creating JobFlow using aws-conn-id: aws_default, emr-conn-id: emr_default
[2019-08-21 05:39:42,981] \{logging_mixin.py:95} INFO - [[34m2019-08-21
05:39:42,980[0m] \{[34mconnection.py:[0m296} ERROR[0m - Expecting property
name enclosed in double quotes: line 1 column 2 (char 1)[0m
[31mTraceback (most recent call last):
File
"/home/ubuntu/.local/lib/python2.7/site-packages/airflow/models/connection.py",
line 294, in extra_dejson
obj = json.loads(self.extra)
File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
return _default_decoder.decode(s)
File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name enclosed in double quotes: line 1 column 2
(char 1)[0m
[2019-08-21 05:39:42,982] \{logging_mixin.py:95} INFO - [[34m2019-08-21
05:39:42,981[0m] \{[34mconnection.py:[0m297} ERROR[0m - Failed parsing the
json for conn_id [1maws_default[0m[0m
[2019-08-21 05:39:43,054] \{taskinstance.py:1047} ERROR - Parameter validation
failed:
Unknown parameter in input: "TerminationProtected", must be one of: Name,
LogUri, AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps,
BootstrapActions, SupportedProducts, NewSupportedProducts, Applications,
Configurations, VisibleToAllUsers, JobFlowRole, ServiceRole, Tags,
SecurityConfiguration, AutoScalingRole, ScaleDownBehavior, CustomAmiId,
EbsRootVolumeSize, RepoUpgradeOnBoot, KerberosAttributes
Unknown parameter in input: "KeepJobFlowAliveWhenNoSteps", must be one of:
Name, LogUri, AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps,
BootstrapActions, SupportedProducts, NewSupportedProducts, Applications,
Configurations, VisibleToAllUsers, JobFlowRole, ServiceRole, Tags,
SecurityConfiguration, AutoScalingRole, ScaleDownBehavior, CustomAmiId,
EbsRootVolumeSize, RepoUpgradeOnBoot, KerberosAttributes
Unknown parameter in input: "Ec2SubnetId", must be one of: Name, LogUri,
AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps, BootstrapActions,
SupportedProducts, NewSupportedProducts, Applications, Configurations,
VisibleToAllUsers, JobFlowRole, ServiceRole, Tags, SecurityConfiguration,
AutoScalingRole, ScaleDownBehavior, CustomAmiId, EbsRootVolumeSize,
RepoUpgradeOnBoot, KerberosAttributes
Unknown parameter in input: "Ec2KeyName", must be one of: Name, LogUri,
AdditionalInfo, AmiVersion, ReleaseLabel, Instances, Steps, BootstrapActions,
SupportedProducts, NewSupportedProducts, Applications, Configurations,
VisibleToAllUsers, JobFlowRole, ServiceRole, Tags, SecurityConfiguration,
AutoScalingRole, ScaleDownBehavior, CustomAmiId, EbsRootVolumeSize,
RepoUpgradeOnBoot, KerberosAttributes}}
h3. {{*CORRESPONDING DAG CODE:*}}
{{}}
{noformat}
// code placeholder
airflow_test_json=json.load(open(airflow_home+'/test.json'))
airflow_asset_analytics_creator = EmrCreateJobFlowOperator(
task_id='create_asset_analytics_databricks_test',
job_flow_overrides=airflow_test_json['Job'],
timeout=10,
aws_conn_id='aws_default',
emr_conn_id='emr_default',
dag=dag
)
airflow_asset_analytics_sensor = EmrJobFlowSensor(
task_id='check_asset_analytics_databricks_stable',
job_flow_id="{{
task_instance.xcom_pull('create_asset_analytics_databricks_test',
key='return_value') }}",
aws_conn_id='aws_default',
dag=dag
)
airflow_asset_analytics_sensor.set_upstream(airflow_asset_analytics_creator)
{noformat}
{{}}
Not sure what more details are required,but the exact code worked flawlessly in
v1.10.0.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
