On Wed, Jan 14, 2009 at 1:00 AM, Karen Tracey <kmtra...@gmail.com> wrote: > On Tue, Jan 13, 2009 at 9:05 AM, Russell Keith-Magee > <freakboy3...@gmail.com> wrote: >> >> > So, the latest patch on #8183 (8138alternate-nodoctestxaction.diff) does >> > not >> > enclose doctest runs in a rolled-back transaction. Thus the effects of >> > doctests can still bleed over into subsequent tests. This doesn't >> > actually >> > cause any failures or errors when running the Django test suite, as far >> > as >> > I've seen. I'm inclined to think it's an OK alternative at least for >> > the >> > short term, but would be interested in hearing other opinions. >> >> I can't say this concerns me particularly. Doctests are nice and all, >> but they aren't a particularly sophisticated testing option. IMHO if >> you have complex testing requirements, you should probably be looking >> at moving beyond doctests. > > OK, so that sounds like one vote for leaving things as they are for > doctests, that is with no rolled-back transaction cleaning up after them. > >> [snip] >> >> >> >> >2 - Think some more about TransactionTestCase. This isn't used by any of >> > the Django tests but I think needs to be provided in case anyone has >> > tests >> > that really do need to make use of transactions internally. I think, >> > though, there is currently a bit of a problem with mixing >> > TransactionTestCases with TestCases since the former clears the DB on >> > entry >> > but not afterward, so its effects are not cleaned up, while TestCases >> > assume >> > the DB is clean on entry. >> >> I agree that the TransactionTestCase needs to be provided, even if we >> aren't using it. >> >> The mixing of TransactionTestCase and TestCase doesn't worry me so >> much; using doctests and TestCase requires that you are aware of the >> way that they interact; this just introduces a different set of >> expectations that we need to manage. >> >> For me, a bigger concern is the change in behaviour of TestCase. >> Moving to a transaction-based TestCase needs to be a zero-impact >> operation, but under the current patch, it isn't. With the existing >> code, a TestCase could assume that the database was clean at the start >> of each test; this isn't the case after applying this patch. > > OK, so what to do? Sources of pollution on entry to a TestCase using the > rollback approach are previously run doctests and (given the way > TransactionTestCase is currently written) previously run > TransactionTestCases. The latter could be dealt with by modifying > TransactionTestCase to flush/syncdb after the run in addition to (or instead > of, if we manage to guarantee clean DB on entry) before the run. That > leaves the doctests again. Some (not fully thought-out) ideas for dealing > with them: > > 1 - Reconsider the idea of cleaning up after each doctest. > > 2 - I've not looked at the test discovery/running machinery at all. Maybe > there is a way to modify it to run all unit tests, then all doctests instead > of interspersing them as it currently does? That way we could guarantee > clean DB on entry to unit tests because no doctests would have been run > before the unit tests. > > 3 - Instead of changing TestCase to use the rollback approach and providing > an opt-out in the form of TransactionTestCase, leave TestCase as it is and > provide a new (e.g. LightningFastTestCase) option that uses rollback. > Change Django tests to use the new faster testcase, but require people with > their own tests to make the decision as to whether they want to switch to it > for their own tests. > > Of the three #2 sounds most attractive to me, but I have no idea if it is > feasible. > > #3 would also protect people from being bitten by invalid assumptions (e.g. > primary keys are assigned starting at one) if we switch TestCase to the > rollback approach. Without doing something like #3 I'm not sure we can > really guarantee zero-impact of this change for user-written tests.
#3 sounds like the nicest option to me. #1 introduces even more slowdowns caused by flush operations; #2 might be possible, but I suspect it will be a complex solution to implement. #3 is a simple approach to implement, easy to explain, and opt-in for any existing usage, so it should be zero impact. If I'm wrong on the complexity of #2, it might be preferable, but I don't think it will be easy. Russ %-) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
