[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774273#comment-16774273
]
ASF subversion and git services commented on AIRFLOW-3616:
--
Commit 54aa290798171ac76ec5e470af13b1e14c104930 in airflow's branch
refs/heads/v1-10-test from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=54aa290 ]
[AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
> Fix For: 1.10.3
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16774038#comment-16774038
]
ASF subversion and git services commented on AIRFLOW-3616:
--
Commit 54aa290798171ac76ec5e470af13b1e14c104930 in airflow's branch
refs/heads/v1-10-stable from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=54aa290 ]
[AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
> Fix For: 1.10.3
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16773228#comment-16773228
]
ASF subversion and git services commented on AIRFLOW-3616:
--
Commit 14fa7ba65cd637577d944de5ae7aa6e2028e3c21 in airflow's branch
refs/heads/v1-10-stable from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=14fa7ba ]
[AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772017#comment-16772017
]
ASF subversion and git services commented on AIRFLOW-3616:
--
Commit d80e945f40fb6aaa95c308e6fb18797cad35e5f8 in airflow's branch
refs/heads/master from Kamil Breguła
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d80e945 ]
[AIRFLOW-3616][AIRFLOW-1215] Add aliases for schema with underscore (#4523)
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16772016#comment-16772016
]
ASF GitHub Bot commented on AIRFLOW-3616:
-
Fokko commented on pull request #4523: [AIRFLOW-3616][AIRFLOW-1215] Add aliases
for schema with underscore
URL: https://github.com/apache/airflow/pull/4523
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
> Fix For: 2.0.0
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742392#comment-16742392
]
ASF GitHub Bot commented on AIRFLOW-3616:
-
mik-laj commented on pull request #4523: [AIRFLOW-3616] Add aliases for schema
with underscore
URL: https://github.com/apache/airflow/pull/4523
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-3616\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3616
### Description
- [ ] A comprehensive description is included in JIRA Ticket, include
analysis of other possible solutions.
### Tests
- [x] Add test_connection_from_uri_with_underscore in
tests.models:ConnectionTest
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp',
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739945#comment-16739945
]
ASF GitHub Bot commented on AIRFLOW-3616:
-
mik-laj commented on pull request #4478: [AIRFLOW-3616] Add aliases for schema
with underscore
URL: https://github.com/apache/airflow/pull/4478
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-3616\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3616
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [ ] A comprehensive description is included in JIRA Ticket, include
analysis of other possible solutions.
### Tests
- [ ] Add test_connection_from_uri_with_underscore in
tests.models:ConnectionTest
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
[jira] [Commented] (AIRFLOW-3616) Connection parsed from URI - unacceptable underscore in schema part
[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739948#comment-16739948
]
ASF GitHub Bot commented on AIRFLOW-3616:
-
mik-laj commented on pull request #4478: [AIRFLOW-3616] Add aliases for schema
with underscore
URL: https://github.com/apache/airflow/pull/4478
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Connection parsed from URI - unacceptable underscore in schema part
>
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Kamil Bregula
>Assignee: Kamil Bregula
>Priority: Major
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
