To add more information to it: We are running our dB on AWS RDS, this is what MySQL say in logs when we switched to mysql-connector:
[Note] Aborted connection 9191 to db: 'connstr' (Got an error reading communication packets) Best Regards, Josip On Monday, July 17, 2017 at 9:47:50 AM UTC+2, josip povreslo wrote: > > Mike, > > Just as an info, mysql-connector is a total NO for us, when I switched to > mysql-connector we saw bigger problems and those happens much more often. > PyMySQL is the last thing to switch to and try. For this processing piece > I'm thinking to replace SQLAlchemy with a direct database driver usage, do > you think that's the catch or this is a more general issue with the drivers > itself? > > Best, > Josip > > On Friday, July 14, 2017 at 11:57:12 PM UTC+2, Mike Bayer wrote: >> >> that "mysqldb" URL is not mysql-connector - it's MySQL-Python (old and >> unmaintained) or mysqlclient (much better, if you have it installed >> and not conflicting w/ MySQL-Python). You'd need to specify >> "mysql+mysqlconnector://" to use that DBAPI, but also I'd recommend >> trying "mysql+pymysql://" to see if you get better results. >> >> as far as "can't connect to server" you should make sure you are not >> surpassing your max_connections limit, though it should be emitting >> different error message for that. >> >> The Queue.Empty error is not the "error", it has to do with how the >> QueuePool works internally to test that the pool is empty, and Python >> 3 can't help but report every exception in the chain. The error is >> the can't connect to server. >> >> >> >> On Fri, Jul 14, 2017 at 5:41 PM, josip povreslo >> <josip.p...@gmail.com> wrote: >> > In addition to the previous email, what I do is the following: >> > >> > When request comes in: >> > self.mysql_engine = >> > >> create_engine('mysql+mysqldb://{}:{}@{}/{}'.format(self.db_params['dbuser'], >> >> > self.db_params['dbpass'], self.db_params['db_hosts'][db_host_index], >> > self.db_params['dbname']), pool_recycle=3600, >> > connect_args={'connect_timeout': 2}) >> > >> > And then when need to interact with the dB: >> > >> > connection = self.mysql_engine.connect() >> > connection.execute("raw query") >> > connection.close() >> > >> > >> > Best Regards, >> > Josip >> > >> > On Friday, July 14, 2017 at 11:36:26 PM UTC+2, josip povreslo wrote: >> >> >> >> Hello, >> >> >> >> As mentioned in the subject, we have the following packages installed: >> >> >> >> Python 3.5 >> >> mysql-connector==2.1.6 >> >> SQLAlchemy==1.1.11 >> >> >> >> Our service is working until we get a bit more traffic (40 - 50 >> req/min), >> >> although I'm still not 100% if it's a pattern, however, then we get >> this >> >> error: >> >> >> >> Traceback (most recent call last): >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 1122, in _do_get >> >> return self._pool.get(wait, self._timeout) >> >> File >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/queue.py", >> line >> >> 145, in get >> >> raise Empty >> >> sqlalchemy.util.queue.Empty >> >> During handling of the above exception, another exception occurred: >> >> Traceback (most recent call last): >> >> File >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/base.py", >> line >> >> 2147, in _wrap_pool_connect >> >> return fn() >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 328, in unique_connection >> >> return _ConnectionFairy._checkout(self) >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 766, in _checkout >> >> fairy = _ConnectionRecord.checkout(pool) >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 516, in checkout >> >> rec = pool._do_get() >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 1138, in _do_get >> >> self._dec_overflow() >> >> File >> >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/langhelpers.py", >> >> line 66, in __exit__ >> >> compat.reraise(exc_type, exc_value, exc_tb) >> >> File >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/util/compat.py", >> line >> >> 187, in reraise >> >> raise value >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 1135, in _do_get >> >> return self._create_connection() >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 333, in _create_connection >> >> return _ConnectionRecord(self) >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 461, in __init__ >> >> self.__connect(first_connect_check=True) >> >> File "/usr/local/lib64/python3.5/site-packages/sqlalchemy/pool.py", >> line >> >> 651, in __connect >> >> connection = pool._invoke_creator(self) >> >> File >> >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/strategies.py", >> >> line 105, in connect >> >> return dialect.connect(*cargs, **cparams) >> >> File >> >> >> "/usr/local/lib64/python3.5/site-packages/sqlalchemy/engine/default.py", >> >> line 393, in connect >> >> return self.dbapi.connect(*cargs, **cparams) >> >> File "/usr/local/lib64/python3.5/site-packages/MySQLdb/__init__.py", >> >> line 81, in Connect >> >> return Connection(*args, **kwargs) >> >> File >> "/usr/local/lib64/python3.5/site-packages/MySQLdb/connections.py", >> >> line 191, in __init__ >> >> super(Connection, self).__init__(*args, **kwargs2) >> >> _mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL >> server >> >> on 'dbhost_obviously' (4)") >> >> >> >> We haven't been seeing this error up until recently when we ported our >> >> service to Python 3.5. I've read most of the articles related to this >> queue >> >> empty error and most of those are from '12 or '13 and mentioning >> immature >> >> mysql drivers. Is this still the case, should we search for an >> alternative >> >> to mysql-connector or alternative to something else? Any help or >> guidance is >> >> appreciated! >> >> >> >> Thanks! >> >> >> >> Best, >> >> Josip >> > >> > -- >> > SQLAlchemy - >> > The Python SQL Toolkit and Object Relational Mapper >> > >> > http://www.sqlalchemy.org/ >> > >> > To post example code, please provide an MCVE: Minimal, Complete, and >> > Verifiable Example. See http://stackoverflow.com/help/mcve for a full >> > description. >> > --- >> > You received this message because you are subscribed to the Google >> Groups >> > "sqlalchemy" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an >> > email to sqlalchemy+...@googlegroups.com. >> > To post to this group, send email to sqlal...@googlegroups.com. >> > Visit this group at https://groups.google.com/group/sqlalchemy. >> > For more options, visit https://groups.google.com/d/optout. >> > -- SQLAlchemy - The Python SQL Toolkit and Object Relational Mapper http://www.sqlalchemy.org/ To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description. --- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.