Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 edited a comment on the discussion: sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' thank you for your reply. I was not aware of that information. In my case, I just downgraded Pandas to 2.1.4 (from 2.2.3) but in the future I will follow the guidelines from that article. GitHub link: https://github.com/apache/airflow/discussions/50411#discussioncomment-13147216 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 edited a comment on the discussion: sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' To those who are interested: In my case, I just downgraded Pandas to 2.1.4 (from 2.2.3) and it worked! GitHub link: https://github.com/apache/airflow/discussions/50411#discussioncomment-13147217 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 closed a discussion: sqlalchemy & pandas don't work
together: 'Connection' object has no attribute 'cursor'
### Error
```python
UserWarning: pandas only supports SQLAlchemy connectable (engine/connection)
or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are
not tested. Please consider using SQLAlchemy.
df = pd.read_sql(query, con=connection)
❌ Query failed: 'Connection' object has no attribute 'cursor'
```
df = pd.read_sql(query, con=connection)
### Reproducible Example
```python
pandas~=2.2.3
requests~=2.32.3
apache-airflow~=2.10.5
SQLAlchemy~=1.4.54
import pandas as pd
from sqlalchemy import create_engine
# Connection string
conn_str = (
"mssql+pyodbc://extl:xxx@serverIP/DB_name"
"?driver=ODBC+Driver+17+for+SQL+Server"
)
# Create the SQLAlchemy engine
engine = create_engine(conn_str)
# Sample SQL query
query = "SELECT TOP 1 * FROM dbo.country"
# Use SQLAlchemy's connection object to execute the query
try:
with engine.connect() as connection:
df = pd.read_sql(query, con=connection)
print(df)
except Exception as e:
print("❌ Query failed:", e)
```
### Next steps?
✅ pandas==2.2.3 and SQLAlchemy==1.4.54 are compatible with each other.
✅ They are also compatible with apache-airflow==2.10.5, which officially
supports SQLAlchemy>=1.4,<2.
So, I can't upgrade SQLAlchemy as apache-airflow~=2.10.5 will not work then.
What would you suggest?
Here is related issue I found:
https://github.com/pandas-dev/pandas/issues/57053
GitHub link: https://github.com/apache/airflow/discussions/50411
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 added a comment to the discussion: sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' thank you for your reply. I was not aware of that information. In my case, I just downgraded Pandas to 2.1.4 (from 2.2.3) and it worked! GitHub link: https://github.com/apache/airflow/discussions/50411#discussioncomment-13147217 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 added a comment to the discussion: sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' thank you for your reply. I was not aware of that information. In my case, I just downgraded Pandas to 2.1.4 (from 2.2.3) and it worked! GitHub link: https://github.com/apache/airflow/discussions/50411#discussioncomment-13147216 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user potiuk added a comment to the discussion: sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' We are working on supporting SQL Alchemy 2 some time later in Airflow 3. Best thing is to upgrade to it when it's ready. - if I understand your problem properly. If you have problem with conflicting requirements you can use one of the approaches described in https://airflow.apache.org/docs/apache-airflow/2.10.5/best-practices.html#handling-conflicting-complex-python-dependencies GitHub link: https://github.com/apache/airflow/discussions/50411#discussioncomment-13120106 This is an automatically sent email for commits@airflow.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
Re: [D] sqlalchemy & pandas don't work together: 'Connection' object has no attribute 'cursor' [airflow]
GitHub user klionsky123 edited a discussion: sqlalchemy & pandas don't work
together: 'Connection' object has no attribute 'cursor'
### Error
```python
UserWarning: pandas only supports SQLAlchemy connectable (engine/connection)
or database string URI or sqlite3 DBAPI2 connection. Other DBAPI2 objects are
not tested. Please consider using SQLAlchemy.
df = pd.read_sql(query, con=connection)
❌ Query failed: 'Connection' object has no attribute 'cursor'
```
df = pd.read_sql(query, con=connection)
### Reproducible Example
```python
pandas~=2.2.3
requests~=2.32.3
apache-airflow~=2.10.5
SQLAlchemy~=1.4.54
import pandas as pd
from sqlalchemy import create_engine
# Connection string
conn_str = (
"mssql+pyodbc://extl:xxx@serverIP/DB_name"
"?driver=ODBC+Driver+17+for+SQL+Server"
)
# Create the SQLAlchemy engine
engine = create_engine(conn_str)
# Sample SQL query
query = "SELECT TOP 1 * FROM dbo.country"
# Use SQLAlchemy's connection object to execute the query
try:
with engine.connect() as connection:
df = pd.read_sql(query, con=connection)
print(df)
except Exception as e:
print("❌ Query failed:", e)
```
### Next steps?
✅ pandas==2.2.3 and SQLAlchemy==1.4.54 are compatible with each other.
✅ They are also compatible with apache-airflow==2.10.5, which officially
supports SQLAlchemy>=1.4,<2.
So, I can't upgrade SQLAlchemy as apache-airflow~=2.10.5 will not work then.
What would you suggest?
Here is related issue I found:
https://github.com/pandas-dev/pandas/issues/57053
GitHub link: https://github.com/apache/airflow/discussions/50411
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscr...@airflow.apache.org
