I would just be scared that some minor issues are different between the database implementations - therefore some test that would work in the tests and during development, doesn't work in production.
I usually try to use the same things in production and development (or as close as possible). That being said - Tests that take 300 minutes is just too much :-) We have around 750 tests are it takes around 5 minutes to run - Something that I find to be too long still.... Regards, Andréas Den lör 15 sep. 2018 kl 02:29 skrev Mike Dewhirst <mi...@dewhirst.com.au>: > +1 Andréas > > One of my projects runs (currently) 1,248 tests using SQLite3 in 72 > minutes on my local Windows 10 dev laptop. That laptop has both a SSD > and a hard disk. Foolishly I opted to use the SSD for software > installations and the hard disk for development and thus the tests. I > was concerned that overexercising the SSD might wear it out. I've since > been advised that I shouldn't worry so one of these days I'll reorganise > things and maybe see that 72 minutes drop somewhat. > > However, those same tests take 285 minutes on a headless PC running > Ubuntu 16.04 LTS server with Postgres 9.6 > > There are only one or two tests which generate slightly different error > messages between SQLite3 and Postgres and that is trivial to manage. > > Most tests are database heavy, requiring queries and calling/testing > model methods. > > Consequently, I use - and recommend - SQLite for dev tests and Postgres > for deployment testing. > > Cheers > > Mike > > On 15/09/2018 1:45 AM, Andréas Kühne wrote: > > Hi, > > > > Just my 5 cents. I think you are doing the tests wrong. I don't > > believe that doing testing against hard coded values is at all correct > > - and it isn't actually that hard to change the tests to a simpler > > way. The values of the PK's aren't really necessary for your test to > > be true either - how does that translate to a real use case? You > > should probably check for A value in the pk field, but not a specific > > value, because that doesn't result in any added value for your customer? > > > > Also changing the way django runs tests feels like working against the > > framework rather than with it? I would probably much prefer changing > > the tests than changing the way the framework runs my tests.... > > Another issue you may face is if Django changes the underlying code, > > then you will get strange failures as well... > > > > I don't think that 70 tests is that much to change either - we work > > with a project that could fail a considerable amount of tests during a > > refactor - and then we need to fix them. The same goes here I think - > > you did a change to the infrastructure that made the tests invalid - > > rewrite the tests :-) > > > > Best regards, > > > > Andréas > > > > > > Den fre 14 sep. 2018 kl 17:32 skrev Hezi Halpert <chez...@gmail.com > > <mailto:chez...@gmail.com>>: > > > > I would like to share with you an issue we encounter while moving > > from sqlite to postgres with heavily use of Django testing. > > > > We have Django app with ~700 tests. Most of them accessing > > database. We recently migrated the database from sqlite to postgres. > > Many of our tests were written in a way that compares actual pk’s > > (hard-coded pks or just file/json comparisons) . Since Django > > testing on sqlite (testcases.TestCase class) creates in-memory > > database (which is being reseted every unit test by default), we > > never had a problem with it. > > However, Django TestCase on postgres create completely another > > test db which preserves the pk sequences between different tests. > > And since many of our tests were written in a way that compares > > actual pk’s they all start fail - depends on the exact tests > > execution order. > > Even tests which expect some pk and are were not failed yet, can > > potentially failed in the future - depends on adding/editing other > > tests which may change the db sequence > > > > We consider the following solutions: > > > > 1. Move to TransactionTestCase (instead of TestCase) and use > > “reset_sequences = True” flag. Cons: > > TransactionTestCase reduces performance dramatically (~4 times > > longer in some of the tests) > > 2. Refactor all failed tests: remove all hard-coded references to > > the pk. Cons: Require much Dev effort (we had more then 70 > > such tests) > > 3. Route the database in settings.py such it will use sqlite > > instead of postgres when running tests. Cons: It will not > > actually test the real scenarios - not an option > > 4. Combine reset_sequences flag with TestCase in our own version > > to TestCase: OurTestCase class and make everything to inherit > > from it. This is the option we finally decided of. See below. > > > > > > fromdjango.test import TestCase, testcases > > > > class OurTestCase(TestCase): > > reset_sequences =True def _fixture_setup(self): > > for db_namein self._databases_names(include_mirrors=False): > > if self.reset_sequences: > > self._reset_sequences(db_name) > > if self.fixtures: > > call_command('loaddata', *self.fixtures, > **{'verbosity':0,'database': db_name}) > > if not testcases.connections_support_transactions(): > > self.setUpTestData() > > return super(TestCase,self)._fixture_setup() > > self.atomics =self._enter_atomics() > > > > Another problem of these kind of tests is the default ordering > > assumption of Django which changes significantly between postgres > > and sqlite when testing. > > Therefore, models included in such tests must have a hint for > > Django regarding the default ordering retrieval. > > Our solution was to make all models inherit from > > DexterModelDefaultOrder (below) > > > > > > class DexterModelDefaultOrder(models.Model): > > class Meta: > > abstract =True ordering = ['id'] > > > > I hope it (will) help someone > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Django users" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to django-users+unsubscr...@googlegroups.com > > <mailto:django-users+unsubscr...@googlegroups.com>. > > To post to this group, send email to django-users@googlegroups.com > > <mailto:django-users@googlegroups.com>. > > Visit this group at https://groups.google.com/group/django-users. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/django-users/ffcd3cc9-c3be-44ba-9665-a4ded5fed492%40googlegroups.com > > < > https://groups.google.com/d/msgid/django-users/ffcd3cc9-c3be-44ba-9665-a4ded5fed492%40googlegroups.com?utm_medium=email&utm_source=footer > >. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > > You received this message because you are subscribed to the Google > > Groups "Django users" group. > > To unsubscribe from this group and stop receiving emails from it, send > > an email to django-users+unsubscr...@googlegroups.com > > <mailto:django-users+unsubscr...@googlegroups.com>. > > To post to this group, send email to django-users@googlegroups.com > > <mailto:django-users@googlegroups.com>. > > Visit this group at https://groups.google.com/group/django-users. > > To view this discussion on the web visit > > > https://groups.google.com/d/msgid/django-users/CAK4qSCeeyDaUnTb95uO_LpdkLknS-V6tWtm8q3%2BY58FgEs_AnQ%40mail.gmail.com > > < > https://groups.google.com/d/msgid/django-users/CAK4qSCeeyDaUnTb95uO_LpdkLknS-V6tWtm8q3%2BY58FgEs_AnQ%40mail.gmail.com?utm_medium=email&utm_source=footer > >. > > For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "Django users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to django-users+unsubscr...@googlegroups.com. > To post to this group, send email to django-users@googlegroups.com. > Visit this group at https://groups.google.com/group/django-users. > To view this discussion on the web visit > https://groups.google.com/d/msgid/django-users/bf694e3d-2916-bfe9-0335-55deb2e6e3d8%40dewhirst.com.au > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAK4qSCfDAMDaJ-5KqEG_isydLTiXRCfY1qg3yi1Mretw0tJC1A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
