Re: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
Regarding the Postgres in production and SQLite in testing differences, we've previously had problems with Postgres's Array - which doesn't exist in SQLite. Also watch out for the differences between how they interpret GROUP BY and DISTINCT - I've had this problem only today where a complicated query I wrote and tested in unittest (SQLite) doesn't work on development localhost (Postgres). Alex, thanks very much for this blog post - very helpful. I was just talking about this strategy with my colleague today. I've previously switch over to Postgres in test to address problems with Array - however I had to bring the project back over again because of speed issues - it was taking far too long to do a set up and tear down of the DB for each test. It looks like your solution will be much quicker than our previous one because of your strategy with transactions. Cheers, James On Wednesday, September 11, 2013 6:12:07 PM UTC+1, Alex Grönholm wrote: I wrote a blog post on this very topic recently: http://alextechrants.blogspot.fi/2013/08/unit-testing-sqlalchemy-apps.html tiistai, 10. syyskuuta 2013 19.43.35 UTC+3 Toph Burns kirjoitti: Could you use an in-memory, sqlite db for your testing? For our applications, we have an initialization function that loads the database connection strings from a config (.ini) file, passing those on to create_engine. In production it's a postgresql connection string, for test, it's a sqlite:///:memory:' Toph Burns | Software Engineer 5885 Hollis St. Suite 100 Emeryville, CA 94608 510-597-4797 bu...@amyris.com -- *From:* sqlal...@googlegroups.com [sqlal...@googlegroups.com] on behalf of Michel Albert [exh...@gmail.com] *Sent:* Tuesday, September 10, 2013 1:46 AM *To:* sqlal...@googlegroups.com *Subject:* [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative. I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com. To post to this group, send email to sqlal...@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
However, be aware of differences between PostgreSQL and sqlite. For example sqlite does not support recursive CTEs. But I am sure there's more. Ladislav Lenart On 10.9.2013 18:43, Toph Burns wrote: Could you use an in-memory, sqlite db for your testing? For our applications, we have an initialization function that loads the database connection strings from a config (.ini) file, passing those on to create_engine. In production it's a postgresql connection string, for test, it's a sqlite:///:memory:' Toph Burns | Software Engineer 5885 Hollis St. Suite 100 Emeryville, CA 94608 510-597-4797 bu...@amyris.com *From:* sqlalchemy@googlegroups.com [sqlalchemy@googlegroups.com] on behalf of Michel Albert [exh...@gmail.com] *Sent:* Tuesday, September 10, 2013 1:46 AM *To:* sqlalchemy@googlegroups.com *Subject:* [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative. I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
I wrote a blog post on this very topic recently: http://alextechrants.blogspot.fi/2013/08/unit-testing-sqlalchemy-apps.html tiistai, 10. syyskuuta 2013 19.43.35 UTC+3 Toph Burns kirjoitti: Could you use an in-memory, sqlite db for your testing? For our applications, we have an initialization function that loads the database connection strings from a config (.ini) file, passing those on to create_engine. In production it's a postgresql connection string, for test, it's a sqlite:///:memory:' Toph Burns | Software Engineer 5885 Hollis St. Suite 100 Emeryville, CA 94608 510-597-4797 bu...@amyris.com javascript: -- *From:* sqlal...@googlegroups.com javascript: [sqlal...@googlegroups.comjavascript:] on behalf of Michel Albert [exh...@gmail.com javascript:] *Sent:* Tuesday, September 10, 2013 1:46 AM *To:* sqlal...@googlegroups.com javascript: *Subject:* [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative. I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com javascript:. To post to this group, send email to sqlal...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
[sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
Re: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
you want to mock the entire relational database at the engine level? that sounds kind of tough?you should mock at the highest level possible in order to test the actual code you're testing.unles you're testing that select * from table returns what you expect, I'm not sure you'd be mocking at that low a level. As for the session, set it up when you need it, then use it. depends on how your application has structured it in the first place, can't say much without specifics. the pattern I use for testing models and use cases is usually against a real database and I do what you see here: http://docs.sqlalchemy.org/en/rel_0_8/orm/session.html#joining-a-session-into-an-external-transaction On Sep 10, 2013, at 4:46 AM, Michel Albert exh...@gmail.com wrote: I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. signature.asc Description: Message signed with OpenPGP using GPGMail
RE: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative.
Could you use an in-memory, sqlite db for your testing? For our applications, we have an initialization function that loads the database connection strings from a config (.ini) file, passing those on to create_engine. In production it's a postgresql connection string, for test, it's a sqlite:///:memory:' Toph Burns | Software Engineer 5885 Hollis St. Suite 100 Emeryville, CA 94608 510-597-4797 bu...@amyris.com From: sqlalchemy@googlegroups.com [sqlalchemy@googlegroups.com] on behalf of Michel Albert [exh...@gmail.com] Sent: Tuesday, September 10, 2013 1:46 AM To: sqlalchemy@googlegroups.com Subject: [sqlalchemy] Unit testing, mocking and dependency injection with SA Declarative. I am trying to wrap my head around how to do Dependency Injection with SQLAlchemy and I am walking in circles. I want to be able to mock out SA for most of my tests. I trust SA and don't want to test serialisation into the DB. I just want to test my own code. So I was thinking to do dependency injection, and mock out SA during testing. But I don't know what to mock out, how and when to set up the session properly, without doing it at the module level (which causes unwanted side-effects only by importing the module). The only solution which comes to mind is to have one singleton which deals with that. But that feels very unpythonic to me and I am wondering if there's a better solution. I also saw that create_engine has an optional module kwarg, which I could mock out. But then SA begins complaining that the return types are not correct. And I don't want to specify return values for every possible db-module call. That's way out of scope of my tests. I am not calling anything on the db-module. That's SA's job, and, as said, I already trust SA. Whenever I work on this I always run into the session_maker initialisation as well. The examples to this on the module level, which I really make me feel uneasy. Any tips? Just prodding myself in the right direction might help me out enough. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
