macpd commented on issue #58724:
URL: https://github.com/apache/airflow/issues/58724#issuecomment-3757094450
I am also experiencing this issue in a fresh (ie not upgraded from a
previous version) install of 3.0.2 installed via helm char.
I have confirmed a PythonOperator is able to read the connection, but that a
PythonVirtualEnvOperator in the same DAG is not:
```
from airflow.sdk import dag, task
@dag(schedule=None)
def test_get_connection_from_postgres_hook():
@task()
def test_named_connection_python_operator():
import logging
from airflow.providers.postgres.hooks.postgres import PostgresHook
logger = logging.getLogger("airflow.task")
postgres_hook =
PostgresHook(postgres_conn_id="named_postgres_connection")
ok, status = postgres_hook.test_connection()
logger.info("ok: %s, status: %s", ok, status)
engine = postgres_hook.get_sqlalchemy_engine()
@task()
def test_postgres_default_connection_python_operator():
import logging
from airflow.providers.postgres.hooks.postgres import PostgresHook
logger = logging.getLogger("airflow.task")
postgres_hook = PostgresHook()
ok, status = postgres_hook.test_connection()
logger.info("ok: %s, status: %s", ok, status)
engine = postgres_hook.get_sqlalchemy_engine()
@task.virtualenv()
def test_postgres_default_connection_python_virtualenv_operator():
import logging
from airflow.providers.postgres.hooks.postgres import PostgresHook
logger = logging.getLogger("airflow.task")
postgres_hook = PostgresHook()
ok, status = postgres_hook.test_connection()
logger.info("ok: %s, status: %s", ok, status)
engine = postgres_hook.get_sqlalchemy_engine()
@task.virtualenv()
def test_postgres_named_connection_python_virtualenv_operator():
import logging
from airflow.providers.postgres.hooks.postgres import PostgresHook
logger = logging.getLogger("airflow.task")
postgres_hook =
PostgresHook(postgres_conn_id="named_postgres_connection")
ok, status = postgres_hook.test_connection()
logger.info("ok: %s, status: %s", ok, status)
engine = postgres_hook.get_sqlalchemy_engine()
test_named_connection_python_operator()
test_postgres_default_connection_python_operator()
test_postgres_default_connection_python_virtualenv_operator()
test_postgres_named_connection_python_virtualenv_operator()
test_get_connection_from_postgres_hook()
```
I confirmed from the CLI that both connections work:
```
$ kubectl exec --namespace airflow-dev svc/airflow-api-server -c api-server
-it -- airflow connections test postgres_default
Retrieving connection: 'postgres_default'
Testing...
Connection success!
$ kubectl exec --namespace airflow-dev svc/airflow-api-server -c api-server
-it -- airflow connections test named_postgres_connection
Retrieving connection: 'named_postgres_connection'
Testing...
Connection success!
```
The 2 @task decorated tasks are able to access both connections, while the
@task.virtualenv tasks fail because they cannot access the connections
<img width="1783" height="505" alt="Image"
src="https://github.com/user-attachments/assets/2d3b82d4-9475-4754-8977-1bb8aac93982";
/>
<img width="2907" height="484" alt="Image"
src="https://github.com/user-attachments/assets/2b9c7af6-dd6f-421e-8d8a-17f0196f0a3b";
/>
I can provide the full logs if needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
