I would just be scared that some minor issues are different between the database implementations - therefore some test that would work in the tests and during development, doesn't work in production.
I usually try to use the same things in production and development (or as close as possible). That being said - Tests that take 300 minutes is just too much :-) We have around 750 tests are it takes around 5 minutes to run - Something that I find to be too long still.... Regards, Andréas Den lör 15 sep. 2018 kl 02:29 skrev Mike Dewhirst <[email protected]>: > +1 Andréas > > One of my projects runs (currently) 1,248 tests using SQLite3 in 72 > minutes on my local Windows 10 dev laptop. That laptop has both a SSD > and a hard disk. Foolishly I opted to use the SSD for software > installations and the hard disk for development and thus the tests. I > was concerned that overexercising the SSD might wear it out. I've since > been advised that I shouldn't worry so one of these days I'll reorganise > things and maybe see that 72 minutes drop somewhat. > > However, those same tests take 285 minutes on a headless PC running > Ubuntu 16.04 LTS server with Postgres 9.6 > > There are only one or two tests which generate slightly different error > messages between SQLite3 and Postgres and that is trivial to manage. > > Most tests are database heavy, requiring queries and calling/testing > model methods. > > Consequently, I use - and recommend - SQLite for dev tests and Postgres > for deployment testing. > > Cheers > > Mike > > On 15/09/2018 1:45 AM, Andréas Kühne wrote: > > Hi, > > > > Just my 5 cents. I think you are doing the tests wrong. I don't > > believe that doing testing against hard coded values is at all correct > > - and it isn't actually that hard to change the tests to a simpler > > way. The values of the PK's aren't really necessary for your test to > > be true either - how does that translate to a real use case? You > > should probably check for A value in the pk field, but not a specific > > value, because that doesn't result in any added value for your customer? > > > > Also changing the way django runs tests feels like working against the > > framework rather than with it? I would probably much prefer changing > > the tests than changing the way the framework runs my tests.... > > Another issue you may face is if Django changes the underlying code, > > then you will get strange failures as well... > > > > I don't think that 70 tests is that much to change either - we work > > with a project that could fail a considerable amount of tests during a > > refactor - and then we need to fix them. The same goes here I think - > > you did a change to the infrastructure that made the tests invalid - > > rewrite the tests :-) > > > > Best regards, > > > > Andréas > > > > > > Den fre 14 sep. 2018 kl 17:32 skrev Hezi Halpert <[email protected] > > <mailto:[email protected]>>: > > > > I would like to share with you an issue we encounter while moving > > from sqlite to postgres with heavily use of Django testing. > > > > We have Django app with ~700 tests. Most of them accessing > > database. We recently migrated the database from sqlite to postgres. > > Many of our tests were written in a way that compares actual pk’s > > (hard-coded pks or just file/json comparisons) . Since Django > > testing on sqlite (testcases.TestCase class) creates in-memory > > database (which is being reseted every unit test by default), we > > never had a problem with it. > > However, Django TestCase on postgres create completely another > > test db which preserves the pk sequences between different tests. > > And since many of our tests were written in a way that compares > > actual pk’s they all start fail - depends on the exact tests > > execution order. > > Even tests which expect some pk and are were not failed yet, can > > potentially failed in the future - depends on adding/editing other > > tests which may change the db sequence > > > > We consider the following solutions: > > > > 1. Move to TransactionTestCase (instead of TestCase) and use > > “reset_sequences = True” flag. Cons: > > TransactionTestCase reduces performance dramatically (~4 times > > longer in some of the tests) > > 2. Refactor all failed tests: remove all hard-coded references to > > the pk. Cons: Require much Dev effort (we had more then 70 > > such tests) > > 3. Route the database in settings.py such it will use sqlite > > instead of postgres when running tests. Cons: It will not > > actually test the real scenarios - not an option > > 4. Combine reset_sequences flag with TestCase in our own version > > to TestCase: OurTestCase class and make everything to inherit > > from it. This is the option we finally decided of. See below. > > > > > > fromdjango.test import TestCase, testcases > > > > class OurTestCase(TestCase): > > reset_sequences =True def _fixture_setup(self): > > for db_namein self._databases_names(include_mirrors=False): > > if self.reset_sequences: > > self._reset_sequences(db_name) > > if self.fixtures: > > call_command('loaddata', *self.fixtures, > **{'verbosity':0,'database': db_name}) > > if not testcases.connections_support_transactions(): > > self.setUpTestData() > > return super(TestCase,self)._fixture_setup() > > self.atomics =self._enter_atomics() > > > > Another problem of these kind of tests is the default ordering > > assumption of Django which changes significantly between postgres > > and sqlite when testing. > > Therefore, models included in such tests must have a hint for > > Django regarding the default ordering retrieval. > > Our solution was to make all models inherit from > > DexterModelDefaultOrder (below) > > > > > > class DexterModelDefaultOrder(models.Model): > > class Meta: > > abstract =True ordering = ['id'] > > > > I hope it (will) help someone > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Django users" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to [email protected] > > <mailto:[email protected]>. > > To post to this group, send email to [email protected] > > <mailto:[email protected]>. > > Visit this group at https://groups.google.com/group/django-users. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/django-users/ffcd3cc9-c3be-44ba-9665-a4ded5fed492%40googlegroups.com > > < > https://groups.google.com/d/msgid/django-users/ffcd3cc9-c3be-44ba-9665-a4ded5fed492%40googlegroups.com?utm_medium=email&utm_source=footer > >. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Django users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to [email protected] > > <mailto:[email protected]>. > > To post to this group, send email to [email protected] > > <mailto:[email protected]>. > > Visit this group at https://groups.google.com/group/django-users. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/django-users/CAK4qSCeeyDaUnTb95uO_LpdkLknS-V6tWtm8q3%2BY58FgEs_AnQ%40mail.gmail.com > > < > https://groups.google.com/d/msgid/django-users/CAK4qSCeeyDaUnTb95uO_LpdkLknS-V6tWtm8q3%2BY58FgEs_AnQ%40mail.gmail.com?utm_medium=email&utm_source=footer > >. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/bf694e3d-2916-bfe9-0335-55deb2e6e3d8%40dewhirst.com.au > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCfDAMDaJ-5KqEG_isydLTiXRCfY1qg3yi1Mretw0tJC1A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
