Am Montag, 18. Januar 2016 12:02:15 UTC+1 schrieb James Schneider: > > A second batch of tests? >> >> I am unsure if I understood you: Do you mean new tests, or running the >> same >> tests again, with a different settings file? >> > > I meant the latter. Sorry for not being clear. > > >> >> If the first: write new tests: How should these tests be different from >> the existing tests? >> >> If the second: Why not run the test with the settings which have >> string_if_invalid enabled in the first run? >> What do you gain by running the tests twice? >> > > The two sets of tests could potentially provide two sets of differing > results. The development setting you want to change is known to negatively > interact with templates used by the Django admin and probably third-party > applications that provide templates, per the link that you provided. With > that setting changed, you are testing an application that intentionally > behaves differently than what will be seen in production. That makes me > uncomfortable. > > Yes, I understand "...behaves differently than what will be seen in production. That makes me uncomfortable." I see it the same.
I would like string_if_invalid to raise an exception immediately. Maybe even in prod, since: "...behaves differently than what will be seen in production. That makes me uncomfortable." Zen of Python: > Errors should never pass silently. For me, that means running two sets of tests. One of them will closely > match the production settings (probably the settings file you're using > now), and the other can be tweaked to potentially catch edge-cases or other > uncommon bugs that you can/have run in to, or other failure points that may > not be easily exposed by the standard settings file. Perhaps the second > customized set of tests may not be something that you build into your CI, > but may be part of a manual examination performed at various milestones. It > would depend on your workflow and CI process, and how often you think these > particular bugs may present themselves. Maybe only as a pre-release final > check? > > We have no "pre-release final check/test". We run the whole test-suite twice daily. This is done automatically be jenkins. We try to automate as much as possible. Manual examination is possible, but who wants to do this? I would like to have it automated. All tests run in around 10 minutes. I guess we will see bugs like four times per year. With "like this" I mean: * bugs caused by the template engine returning the empy string instead of an exception * bugs which are not detected by CI and get into production > Or, if you don't use the admin or any 3rd party applications, you may be > safe in enabling it. I personally wouldn't unless I was also doing so in > production. > > Yes, dev and prod should not be much different Thank you for your input. Regards, Thomas -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To post to this group, send email to django-users@googlegroups.com. Visit this group at https://groups.google.com/group/django-users. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/892db691-3437-461e-80bc-1396ef854834%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
