[sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Re: django user (using django ORM) + sqlalchemy for other db tables
Thanks for the advice -- your recommendations against this configuration were a surprise to me... It's making me rethink what I want (and how much I want it). I'll post this as a comment to the first stackoverflow question so others are made aware. On Monday, September 9, 2013 10:17:10 PM UTC+8, Mauricio de Abreu Antunes wrote: The major problem is: everything in Django is mapped to the ORM. Even the sessions. Sorry for being negative but that is my way to understand this. 2013/9/9 Jonathan Vanasco jona...@findmeon.com javascript: Honestly, I wouldn't do this. Django has a lot of magic under the hood, and it's ORM does some very specific things to make this magic happen. It's not just the auth, it's how everything is structured in the database and how the app integrates with the database. You're likely to break things and be miserable. Django , Rails, etc are all-in frameworks -- they provide a stack with several decisions made for you; it's all or nothing. I'd personally suggest you either: - Continue using Django with their ORM. Create a secondary model that uses SqlAlchemy to reflect the Django mapping. That will let you use SqlAlchemy to do advanced read queries. - Use a different framework ( Pyramid, Flask, etc ; many have auth plugins that work with SqlAlchemy models ) -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com javascript:. To post to this group, send email to sqlal...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- *Mauricio de Abreu Antunes* * * Github: https://github.com/mauricioabreu Twitter: https://twitter.com/maugzoide -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
you want to mock the entire relational database at the engine level? that sounds kind of tough?you should mock at the highest level possible in order to test the actual code you're testing.unles you're testing that select * from table returns what you expect, I'm not sure you'd be mocking at that low a level. As for the session, set it up when you need it, then use it. depends on how your application has structured it in the first place, can't say much without specifics. the pattern I use for testing models and use cases is usually against a real database and I do what you see here: http://docs.sqlalchemy.org/en/rel_0_8/orm/session.html#joining-a-session-into-an-external-transaction On Sep 10, 2013, at 4:46 AM, Michel Albert exh...@gmail.com wrote: I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [sqlalchemy] Re: django user (using django ORM) + sqlalchemy for other db tables
On Tuesday, September 10, 2013 6:47:41 AM UTC-4, Dennis wrote: Thanks for the advice -- your recommendations against this configuration were a surprise to me... It's making me rethink what I want (and how much I want it). I'll post this as a comment to the first stackoverflow question so others are made aware. https://twitter.com/maugzoide Not sure why its a surprise. There's a reason why you don't see SqlAlchemy used in Django. To clarify -- it's entirely possible to use SqlAlchemy or most other ORMs with Django... it's just that many of the core apps ( like the automatic admin tools , which is why most people choose to use Django in the first place ) are tightly coupled to the stock ORM. By switching the ORM, you lose most of the automatic admin backend functionalities. This is pretty common knowledge, it's listed on their site in a few places, as well as Stack Overflow and Quora. WIthout the admin tools , and ditching the ORM for SqlAlchemy, I don't see much of a point in using Django. You'd probably have an easier time building and maintaining the same project in Flask or Pyramid. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
RE: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
Could you use an in-memory, sqlite db for your testing? For our applications, we have an initialization function that loads the database connection strings from a config (.ini) file, passing those on to create_engine. In production it's a postgresql connection string, for test, it's a sqlite:///:memory:' Toph Burns | Software Engineer 5885 Hollis St. Suite 100 Emeryville, CA 94608 510-597-4797 bu...@amyris.com From: sqlalchemy@googlegroups.com [sqlalchemy@googlegroups.com] on behalf of Michel Albert [exh...@gmail.com] Sent: Tuesday, September 10, 2013 1:46 AM To: sqlalchemy@googlegroups.com Subject: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative. I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] dynamic schema with postgresql
I need to create a new schema with some tables in it whenever a new company
record is added.
Below are my entities (defined with Flask-SqlAlchemy framework extension):
class Company(db.Model):
__tablename__ = 'company'
__table_args__ = {schema:app}
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False)
class Customer(db.Model):
__tablename__ = 'customer'
company_id = db.Column(db.Integer, db.ForeignKey('app.company.id',
onupdate='CASCADE', ondelete='CASCADE'), nullable=False)
id = db.Column(db.Integer, primary_key=True)
code = db.Column(db.String(20), nullable=False)
name = db.Column(db.String(100), nullable=False)
Now when a company is inserted the following code executes:
company_schema = 'c' + str(company_id)
db.session.execute(CreateSchema(company_schema))
db.session.commit()
meta = db.MetaData(bind=db.engine)
for table in db.metadata.tables.items():
if table[1].name not in ('company'):
table[1].tometadata(meta, company_schema)
meta.create_all()
The above will throw an error:
NoReferencedTableError: Foreign key associated with column
'customer.company_id' could not find table 'company'
After some googling, I found that tometadata() changes constraint schema
too. I also found a code,
suggested by Michael Bayer (
http://www.mail-archive.com/sqlalchemy@googlegroups.com/msg13008.html):
def copy_table(table, metadata, schema_map):
args = []
for c in table.columns:
args.append(c.copy())
for c in table.constraints:
if isinstance(c, db.ForeignKeyConstraint):
elem = list(c.elements)[0]
schema = schema_map[elem.column.table.schema]
else:
schema=None
args.append(c.copy(schema=schema))
return db.Table(table.name, metadata, schema=schema_map[table.schema],
*args)
Now I replaced tometadata() with the above function:
copy_table(table[1], meta, {None: schema_name, 'app': 'app'})
print meta.tables.items()
But the error is the same: NoReferencedTableError. However, print clearly
shows that
for company_id constraint, schema hasn't changed:
('c28.customer',
Table('customer',
MetaData(bind=Engine(postgresql://postgres:concept@localhost:5432/clients)),
Column('company_id', Integer(), ForeignKey('app.company.id'),
table=customer, nullable=False),
Column('id', Integer(), table=customer, primary_key=True,
nullable=False),
Column('description', String(length=100), table=customer,
nullable=False), schema='c28'))
I also tried to define ForeignKey as follows, but the error is the same:
company_id = db.Column(db.Integer, db.ForeignKey(Company.__table__.c.id,
onupdate='CASCADE', ondelete='CASCADE'), nullable=False)
Can someone kindly help me with this issue?
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To unsubscribe from this group and stop receiving emails from it, send an email
to sqlalchemy+unsubscr...@googlegroups.com.
To post to this group, send email to sqlalchemy@googlegroups.com.
Visit this group at http://groups.google.com/group/sqlalchemy.
For more options, visit https://groups.google.com/groups/opt_out.
