Re: [sqlalchemy] begin_nested failures starting with 0.7
On Jun 1, 2011, at 7:00 PM, Jon Nelson wrote: > On Wed, Jun 1, 2011 at 4:47 PM, Michael Bayer > wrote: >> metadata.reflect() would close() the connection used for reflection as it >> assumed it was passed an Engine, not a Connection, fixed in r926ee70b67ff. >> Nothing to do with begin_nested() or anything like that. > > Why did this work in 0.6.[1,2,3,4,5,6,7] ? > What's the "right" way to do this? This is a brief summary of metadata.reflect() from 0.5-0.6 (was only introduced in 0.5) def reflect(bind): conn = bind.connect() go_reflect_the_tables(conn) Conspicuously missing is a corresponding close() of the connection. In 0.7 we got all 3000 something tests to pass on Pypy. Pypy uses generational garbage collection, which means it does not garbage collect anything until the GC runs periodically. A half dozen tests against metadata.reflect() and the connection pool would freeze, as each test left a connection hanging open. We'd run out of connections. So here was the fix in 0.7: def reflect(bind): conn = bind.connect() go_reflect_the_tables(conn) conn.close() fixed ! Except it broke your app that happened to be passing a Connection, not an Engine (Connection has a connect() method too so "bind" can be either), and we closed your connection on you. We had no tests for that. Now we do. Fixed again ! -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] begin_nested failures starting with 0.7
On Wed, Jun 1, 2011 at 6:00 PM, Jon Nelson wrote: > On Wed, Jun 1, 2011 at 4:47 PM, Michael Bayer > wrote: >> metadata.reflect() would close() the connection used for reflection as it >> assumed it was passed an Engine, not a Connection, fixed in r926ee70b67ff. >> Nothing to do with begin_nested() or anything like that. > > Why did this work in 0.6.[1,2,3,4,5,6,7] ? Nevermind. . Long day, my bad. Thank you for fixing! -- Jon -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] begin_nested failures starting with 0.7
On Wed, Jun 1, 2011 at 4:47 PM, Michael Bayer wrote: > metadata.reflect() would close() the connection used for reflection as it > assumed it was passed an Engine, not a Connection, fixed in r926ee70b67ff. > Nothing to do with begin_nested() or anything like that. Why did this work in 0.6.[1,2,3,4,5,6,7] ? What's the "right" way to do this? -- Jon -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] begin_nested failures starting with 0.7
metadata.reflect() would close() the connection used for reflection as it
assumed it was passed an Engine, not a Connection, fixed in r926ee70b67ff.
Nothing to do with begin_nested() or anything like that.
On Jun 1, 2011, at 5:10 PM, Jon Nelson wrote:
> I've got a chunk of code that started failed as soon as I started
> testing with 0.7.
> The failure is:
>
> sqlalchemy.exc.StatementError: This Connection is closed '\nCREATE
> TABLE bar (\n\ta TEXT NOT NULL\n)\n\n' None
>
> I'll note that the logging indicates that the connection was returned
> to the pool *un* closed.
> The following code demonstrates the issue.
> I am using postgresql 9.0
>
> Is this a bug?
>
>
>
>
>
> #! /usr/bin/python
> # -*- coding: utf-8 -*-
> import sys
> import logging
> import sqlalchemy as sa
> import sqlalchemy.orm as sa_orm
>
> logger = logging.getLogger({ '__main__': None }.get(__name__, __name__))
>
> def setup_logging():
> logging.basicConfig(stream=sys.stderr, level=logging.WARNING,
> format="%(asctime)s:%(levelname)s:%(name)s:%(message)s")
> logging.getLogger().setLevel(logging.DEBUG)
> logging.getLogger('sqlalchemy').setLevel(logging.DEBUG)
>
> def prep_database(dburi):
> engine_opts = dict()
> engine_opts['url'] = dburi
>
> session_opts = dict(autoflush=False, autocommit=True)
>
> engine = sa.engine_from_config(engine_opts, prefix='')
> factory = sa_orm.sessionmaker(bind=engine, **session_opts)
>
> meta = sa.MetaData()
> dbsession = factory()
>
> return (dbsession, meta)
>
> def main():
> setup_logging()
> dburi = sys.argv[1]
> (sess, meta) = prep_database(dburi)
>
> sess.begin()
> conn = sess.connection()
>
> if 'bar' not in meta.tables:
>sess.begin_nested()
>try:
> meta.reflect(bind=conn, only=['bar'])
>except:
> pass
>sess.rollback() # not a real rollback
>
> if 'bar' not in meta.tables:
>t = sa.Table('bar', meta,
> sa.Column('a', sa.TEXT(), nullable=False),
>)
>t.create(bind=conn)
>
> if 'bar' in meta.tables:
>t = meta.tables['bar']
>t.drop()
>meta.remove(t)
>del t
>
> sess.commit()
>
> if __name__ == '__main__':
> main()
>
> --
> Jon
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalchemy@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] begin_nested failures starting with 0.7
I've got a chunk of code that started failed as soon as I started
testing with 0.7.
The failure is:
sqlalchemy.exc.StatementError: This Connection is closed '\nCREATE
TABLE bar (\n\ta TEXT NOT NULL\n)\n\n' None
I'll note that the logging indicates that the connection was returned
to the pool *un* closed.
The following code demonstrates the issue.
I am using postgresql 9.0
Is this a bug?
#! /usr/bin/python
# -*- coding: utf-8 -*-
import sys
import logging
import sqlalchemy as sa
import sqlalchemy.orm as sa_orm
logger = logging.getLogger({ '__main__': None }.get(__name__, __name__))
def setup_logging():
logging.basicConfig(stream=sys.stderr, level=logging.WARNING,
format="%(asctime)s:%(levelname)s:%(name)s:%(message)s")
logging.getLogger().setLevel(logging.DEBUG)
logging.getLogger('sqlalchemy').setLevel(logging.DEBUG)
def prep_database(dburi):
engine_opts = dict()
engine_opts['url'] = dburi
session_opts = dict(autoflush=False, autocommit=True)
engine = sa.engine_from_config(engine_opts, prefix='')
factory = sa_orm.sessionmaker(bind=engine, **session_opts)
meta = sa.MetaData()
dbsession = factory()
return (dbsession, meta)
def main():
setup_logging()
dburi = sys.argv[1]
(sess, meta) = prep_database(dburi)
sess.begin()
conn = sess.connection()
if 'bar' not in meta.tables:
sess.begin_nested()
try:
meta.reflect(bind=conn, only=['bar'])
except:
pass
sess.rollback() # not a real rollback
if 'bar' not in meta.tables:
t = sa.Table('bar', meta,
sa.Column('a', sa.TEXT(), nullable=False),
)
t.create(bind=conn)
if 'bar' in meta.tables:
t = meta.tables['bar']
t.drop()
meta.remove(t)
del t
sess.commit()
if __name__ == '__main__':
main()
--
Jon
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
