Re: [I] MWAA: psycopg2.OperationalError: SSL connection has been closed unexpectedly [airflow]

2024-12-11 Thread via GitHub


potiuk closed issue #44837: MWAA: psycopg2.OperationalError: SSL connection has 
been closed unexpectedly
URL: https://github.com/apache/airflow/issues/44837


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [I] MWAA: psycopg2.OperationalError: SSL connection has been closed unexpectedly [airflow]

2024-12-11 Thread via GitHub


eladkal commented on issue #44837:
URL: https://github.com/apache/airflow/issues/44837#issuecomment-2535721454

   This seems to be MWAA specific issue. You are discussing about specific 
settings with specific environment.
   I'd suggest to contact MWAA support.
   
   If you have reproduce steps from scratch on Airflow main branch with code 
example that worked on pervious version but isn't working now we can 
investigate, until then it's not clear if there is a bug or it's just something 
with MWAA or your deployment configuration.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



Re: [I] MWAA: psycopg2.OperationalError: SSL connection has been closed unexpectedly [airflow]

2024-12-11 Thread via GitHub


boring-cyborg[bot] commented on issue #44837:
URL: https://github.com/apache/airflow/issues/44837#issuecomment-2535315013

   Thanks for opening your first issue here! Be sure to follow the issue 
template! If you are willing to raise PR to address this issue please do so, no 
need to wait for approval.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



[I] MWAA: psycopg2.OperationalError: SSL connection has been closed unexpectedly [airflow]

2024-12-11 Thread via GitHub


sweetpythoncode opened a new issue, #44837:
URL: https://github.com/apache/airflow/issues/44837

   ### Apache Airflow version
   
   2.10.3
   
   ### If "Other Airflow 2 version" selected, which one?
   
   2.8.1
   
   ### What happened?
   
   `[2024-10-08, 16:38:05 UTC] {{taskinstance.py:2698}} ERROR - Task failed 
with exception
   Traceback (most recent call last):
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 1094, in _commit_impl
   self.engine.dialect.do_commit(self.connection)
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py",
 line 686, in do_commit
   dbapi_connection.commit()
   psycopg2.OperationalError: SSL connection has been closed unexpectedly
   
   
   The above exception was the direct cause of the following exception:
   
   Traceback (most recent call last):
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/models/taskinstance.py",
 line 434, in _execute_task
   with create_session() as session:
 File "/usr/local/lib/python3.11/contextlib.py", line 144, in __exit__
   next(self.gen)
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/utils/session.py",
 line 39, in create_session
   session.commit()
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py",
 line 1454, in commit
   self._transaction.commit(_to_root=self.future)
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/orm/session.py",
 line 839, in commit
   trans.commit()
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 2469, in commit
   self._do_commit()
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 2659, in _do_commit
   self._connection_commit_impl()
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 2630, in _connection_commit_impl
   self.connection._commit_impl()
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 1096, in _commit_impl
   self._handle_dbapi_exception(e, None, None, None, None)
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 2134, in _handle_dbapi_exception
   util.raise_(
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/util/compat.py",
 line 211, in raise_
   raise exception
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/base.py",
 line 1094, in _commit_impl
   self.engine.dialect.do_commit(self.connection)
 File 
"/usr/local/airflow/.local/lib/python3.11/site-packages/sqlalchemy/engine/default.py",
 line 686, in do_commit
   dbapi_connection.commit()
   sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL connection 
has been closed unexpectedly
   
   (Background on this error at: https://sqlalche.me/e/14/e3q8)`
   
   
   Airflow task itself was successful, but it failed on exit and only after ~30 
minutes of runtime, if task runs lower than this time, there are no issues. 
**If i run same task on 2.4.3, all good.**
   
   ### What you think should happen instead?
   
   Task finished without any errros.
   
   ### How to reproduce
   
   MWAA: 2.8.1, 2.10.1.
   Run successful task for 30+ minutes.
   More details: 
https://repost.aws/questions/QU9VhJrACSSdy7zJku4pVO9Q/mwaa-psycopg2-operationalerror-ssl-connection-has-been-closed-unexpectedly
   
   ### Operating System
   
   Aws MWAA
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-amazon  8.16.0
   apache-airflow-providers-celery  3.5.1
   apache-airflow-providers-common-io   1.2.0
   apache-airflow-providers-common-sql  1.10.0
   apache-airflow-providers-databricks  6.0.0
   apache-airflow-providers-ftp 3.7.0
   apache-airflow-providers-google  10.13.1
   apache-airflow-providers-http4.8.0
   apache-airflow-providers-imap3.5.0
   apache-airflow-providers-postgres5.10.0
   apache-airflow-providers-sendgrid3.4.0
   apache-airflow-providers-sftp4.8.1
   apache-airflow-providers-slack   8.5.1
   apache-airflow-providers-sqlite  3.7.0
   apache-airflow-providers-ssh 3.10.0
   apache-airflow-providers-trino   5.6.0
   astronomer-cosmos1.7.1
   
   ### Deployment
   
   Amazon (AWS) MWAA
   
   ### Deployment details
   
   Version: 2.8.1+
   Size: mw1.large
   
   
   ### Anything else?
   
   During the investigation, I checked:
   
   1. new mwaa cluster.
   2. latest mwaa version.
   3. this options:
   ```
   database.sql_alchemy_pool_size 20 
   database.sql_alchemy_pool_recycle 300
   database.sql_alchemy_max_overflow 20
   database.sql_alchemy_pool_pre_ping true
   ```
   
   sql-alchemy-connect-args and all others related to engine + keepal