[
https://issues.apache.org/jira/browse/AIRFLOW-3616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16772016#comment-16772016
]
ASF GitHub Bot commented on AIRFLOW-3616:
-----------------------------------------
Fokko commented on pull request #4523: [AIRFLOW-3616][AIRFLOW-1215] Add aliases
for schema with underscore
URL: https://github.com/apache/airflow/pull/4523
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Connection parsed from URI - unacceptable underscore in schema part
> --------------------------------------------------------------------
>
> Key: AIRFLOW-3616
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3616
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Kamil Bregula
> Assignee: Kamil Bregula
> Priority: Major
> Fix For: 2.0.0
>
>
> There is a problem with creating a new connection based on the state of the
> environment variable if the connection type contains the underscore character.
> If we want to configure a connection based on an environment variable, we
> must create an environment variable. The name of the environment must be
> given according to the scheme:
> {{AIRFLOW_CONN_[CONN_ID]}}
> where {{[CONN_ID]}} is the connection identifier to be used, written in
> upper case.
> The content of the variable defines the connection and is saved in the form
> of URI.
> Defining a URI is complex, but the key is that the connection type is given
> as the schema. There are many possible values to give. but the sample values
> are {{mysql}}, {{postgresql}} or {{google_cloud_platform}}. Unfortunately,
> the last case is not properly handled.
> This is caused by using {{urllib.parse}} to process the value. Unfortunately,
> this module does not support the uppercase character in schema port of URI -
> see below snippets showing the behaviour.
> Since urllib.parse is really there to parse URLs and it is not good for
> parsing non-URL URIs - we should likely use different parser which handles
> more generic URIs.
> Especially that it also creates other problems:
> https://issues.apache.org/jira/browse/AIRFLOW-3615
> Another solution is to create aliases for each connection type with a variant
> that does not contain an unacceptable character. For example
> {{google_cloud_platform => gcp}}. It is worth noting that one alias is
> currently defined - {{postgresql => postgres}}.
> Snippet showing urrlib.parse behaviour:
> Python 3.6.5 (default, Oct 3 2018, 10:03:09)
> Type 'copyright', 'credits' or 'license' for more information
> IPython 7.0.1 – An enhanced Interactive Python. Type '?' for help
> In [1]: from urllib.parse import urlparse
> In [2]:
> urlparse("google_cloud_platform://user:pass@hostname/path?extra_param=extra_param_value")
> Out[2]: ParseResult(scheme='', netloc='',
> path='google_cloud_platform://user:pass@hostname/path', params='',
> query='extra_param=extra_param_value', fragment='')
> In [3]:
> urlparse("gcp://user:pass@hostname/path?extra_param=extra_param_value")
> Out[3]: ParseResult(scheme='gcp', netloc='user:pass@hostname', path='/path',
> params='', query='extra_param=extra_param_value', fragment='')
> Connection parsed from URI - unacceptable underscore in schema part
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
