Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
amoghrajesh closed issue #49881: Airflow3. URI field does not exist for Connection imported from task-sdk. URL: https://github.com/apache/airflow/issues/49881 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
amoghrajesh commented on issue #49881:
URL: https://github.com/apache/airflow/issues/49881#issuecomment-3232188832
@MaksYermak this has been fixed in one of the patch releases (3.0.4) likely
and it works as expected on main too:
```
conn = Connection(
conn_id='test_conn',
conn_type='postgres',
host='localhost',
port=5432,
login='user',
password='pass',
extra='{"ssl": true}'
)
conn.get_uri()
Out[4]:
'postgres://user:pass@localhost:5432/?__extra__=%7B%22ssl%22%3A+true%7D'
```
Closing this issue in that favour. Request you to consume the latest patch
release for Airflow 3 (3.0.5/3.0.6) and it should work
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
amoghrajesh commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2876601210 Cool. Let us know how it goes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
MaksYermak commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2872745176 > [@MaksYermak](https://github.com/MaksYermak) did you get a chance to get around this one? I didn't have time to check @kaxil's solution. I think this solution may suit us, but I am not sure that it will cover all functionality and I want to check it. > [@MaksYermak](https://github.com/MaksYermak) i think we can also try and leverage the get_uri method we expose? https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/definitions/connection.py#L61 This solution does not suit us, because we need to create a `Connection` object. In the current implementation we create a `Connection` object using the `uri` parameter. In the solution which your proposed `get_uri` method exists only for an already created `Connection` object. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
amoghrajesh commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2872469268 @MaksYermak did you get a chance to get around this one? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
amoghrajesh commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2835673892 @MaksYermak i think we can also try and leverage the get_uri method we expose? https://github.com/apache/airflow/blob/main/task-sdk/src/airflow/sdk/definitions/connection.py#L61 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
kaxil commented on issue #49881:
URL: https://github.com/apache/airflow/issues/49881#issuecomment-2835351292
The `Connection` object in Airflow 2.x could use either uri or other args to
construct the object. You can just do the following:
```diff
diff --git
a/providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py
b/providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py
index 8ffdd7c1a4..b5accdcb0f 100644
--- a/providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py
+++ b/providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py
@@ -1118,8 +1118,45 @@ class CloudSQLDatabaseHook(BaseHook):
Connection ID will be randomly generated according to whether it
uses
proxy, TCP, UNIX sockets, SSL.
"""
-uri = self._generate_connection_uri()
-connection = Connection(conn_id=self.db_conn_id, uri=uri)
+if not self.user:
+raise AirflowException("The login parameter needs to be set in
connection")
+if not self.public_ip:
+raise AirflowException("The location parameter needs to be set
in connection")
+if not self.password:
+raise AirflowException("The password parameter needs to be set
in connection")
+if not self.database:
+raise AirflowException("The database parameter needs to be set
in connection")
+
+if self.use_proxy:
+if self.sql_proxy_use_tcp:
+host = "127.0.0.1"
+port = self.sql_proxy_tcp_port
+else:
+host = self.sql_proxy_unique_path
+port = None
+else:
+host = self.public_ip
+port = self.public_port
+
+extra = {}
+if self.use_ssl:
+extra.update({
+"sslmode": "verify-ca",
+"sslcert": self.sslcert,
+"sslkey": self.sslkey,
+"sslrootcert": self.sslrootcert
+})
+
+connection = Connection(
+conn_id=self.db_conn_id,
+conn_type=self.database_type,
+host=host,
+port=port,
+login=self.user,
+password=self.password,
+schema=self.database,
+extra=json.dumps(extra) if extra else None
+)
self.log.info("Creating connection %s", self.db_conn_id)
return connection
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
MaksYermak commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2835301669 > Given that we are doing all this (linked below) to generate URI, we should just simply construct a Connection object in the google provider Sorry I don't understand what you mean `just simply construct a Connection object`, could you please clarify? In `cloud_sql` the code firstly generates a URI, using a function which you shared, and, then, creates a Connection object with this generated URI. I do not understand how we can create a Connection object, using `task-sdk`, with this unique URI when the code does not have the URI field. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
kaxil commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2835254999 Given that we are doing all this (linked below) to generate URI, we should just simply construct a Connection object in the google provider https://github.com/apache/airflow/blob/75140f621961e8e85665ffb058da14fa81791b33/providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py#L1048-L1103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
Re: [I] Airflow3. URI field does not exist for Connection imported from task-sdk. [airflow]
tirkarthi commented on issue #49881: URL: https://github.com/apache/airflow/issues/49881#issuecomment-2835241054 cc: @amoghrajesh @kaxil -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
