Re: replacing __dict__ with an OrderedDict
Am 06.01.2012 12:44, schrieb Peter Otten: [running unit tests in the order of their definition] class Loader(unittest.TestLoader): def getTestCaseNames(self, testCaseClass): """Return a sequence of method names found within testCaseClass sorted by co_firstlineno. """ def first_lineno(name): method = getattr(testCaseClass, name) return method.im_func.__code__.co_firstlineno function_names = super(Loader, self).getTestCaseNames(testCaseClass) function_names.sort(key=first_lineno) return function_names After using this a bit, it works great. I have up to now only found a single problem, and that is that with decorated functions it doesn't even get at the actual line number of the real code, so sorting by that number doesn't work. An example for this is "@unittest.expectedFailure(...)". I can easily ignore this though, just wanted to give this feedback Thanks again! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
In article , Lie Ryan wrote: > >> All serious database has rollback feature when they're available to > >> quickly revert database state in the setUp/cleanUp phase. > > > > I guess MongoDB is not a serious database? > > I guess there are always those oddball cases, but if you choose MongoDB > then you already know the consequences that it couldn't be as easily > unit-tested. And in any case, it is generally a bad idea to unittest > with a database that contains 100 million items, that's for performance > testing. So your point is? My point is that in the real world, what is practical and efficient and sound business is not always what is theoretically correct. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Am 10.01.2012 13:31, schrieb Lie Ryan: While it is possible to replace __dict__ with OrderedDict and it is possible to reorder the test, those are not his original problem, and the optimal solution to his original problem differs from the optimal solution to what he think he will need. Oh, and you know what is optimal in his environment? You really think you know better what to do based on the little information provided? I had said this before and I'm saying it again: the problem is a test result displaying issue, not testing order issue. There are good reasons for not reordering the results of the tests that would be sacrificed by reordering them afterwards. If you'd just step off your high horse you might actually learn something instead of just pissing people off. Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Tue, 2012-01-10 at 09:05 -0500, Roy Smith wrote: > > I guess MongoDB is not a serious database? That's opening up a can of worms ;) ... anyway, cassandra is far better. Tim -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/11/2012 01:05 AM, Roy Smith wrote: In article, Lie Ryan wrote: On 01/10/2012 12:05 PM, Roy Smith wrote: Somewhat more seriously, let's say you wanted to do test queries against a database with 100 million records in it. You could rebuild the database from scratch for each test, but doing so might take hours per test. Sometimes, real life is just*so* inconvenient. All serious database has rollback feature when they're available to quickly revert database state in the setUp/cleanUp phase. I guess MongoDB is not a serious database? I guess there are always those oddball cases, but if you choose MongoDB then you already know the consequences that it couldn't be as easily unit-tested. And in any case, it is generally a bad idea to unittest with a database that contains 100 million items, that's for performance testing. So your point is? -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
In article , Lie Ryan wrote: > On 01/10/2012 12:05 PM, Roy Smith wrote: > > Somewhat more seriously, let's say you wanted to do test queries against > > a database with 100 million records in it. You could rebuild the > > database from scratch for each test, but doing so might take hours per > > test. Sometimes, real life is just*so* inconvenient. > > All serious database has rollback feature when they're available to > quickly revert database state in the setUp/cleanUp phase. I guess MongoDB is not a serious database? -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/10/2012 03:59 AM, Ulrich Eckhardt wrote: There is another dependency and that I'd call a logical dependency. This occurs when e.g. test X tests for an API presence and test Y tests the API behaviour. In other words, Y has no chance to succeed if X already failed. Unfortunately, there is no way to express this relation, there is no "@unittest.depends(test_X)" to decorate test_Y with (Not yet!). The skipIf decorator exists precisely for this purpose. Generally, testing availability of resources (like existence of an API) should be done outside of the testing code. In other words, test_X should never be a test in the first place, it should be part of the setting up of the tests; the tests themselves should be completely independent of each other. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/10/2012 12:16 AM, Ulrich Eckhardt wrote: Am 09.01.2012 13:10, schrieb Lie Ryan: I was just suggesting that what the OP thinks he wants is quite likely not what he actually wants. Rest assured that the OP has a rather good idea of what he wants and why, the latter being something you don't know, because he never bothered to explain it and you never asked. Please don't think he's an idiot just because he wants something that doesn't make sense to you. The OP explained the "why" clearly in his first post, he wanted to see his test results ordered in a certain way to make debugging easier, to quote the OP: """ ... I just want to take the first test that fails and analyse that instead of guessing the point to start debugging from the N failed tests. """ and then he goes on concluding that he need to reorder the tests itself and to replace __dict__ with OrderedDict. While it is possible to replace __dict__ with OrderedDict and it is possible to reorder the test, those are not his original problem, and the optimal solution to his original problem differs from the optimal solution to what he think he will need. I had said this before and I'm saying it again: the problem is a test result displaying issue, not testing order issue. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/10/2012 12:05 PM, Roy Smith wrote: Somewhat more seriously, let's say you wanted to do test queries against a database with 100 million records in it. You could rebuild the database from scratch for each test, but doing so might take hours per test. Sometimes, real life is just*so* inconvenient. All serious database has rollback feature when they're available to quickly revert database state in the setUp/cleanUp phase. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 1/9/2012 8:05 PM, Roy Smith wrote: In article<11jrt8-l32@satorlaser.homedns.org>, Ulrich Eckhardt wrote: Some people advocate that the test framework should intentionally randomize the order, to flush out inter-test dependencies that the author didn't realize existed (or intend). If you now happen to influence one test with another and the next run randomizes the tests differently, you will never see the fault again. Without this reproducability, you don't gain anything but the bad stomach feeling that something is wrong. The standard solution to that is to print out the PRNG initialization state and provide a way in your test harness to re-initialize it to that state. I've done things like that in test scenarios where it is difficult or impossible to cover the problem space deterministically. Your unfortunate case is where test X creates persistent state that must be present in order for test X+1 to produce meaningful results. This kind of dependency obviously blows, as it means you can't debug test X+1 separately. I'd call this operational dependency. This kind of dependency is IMHO a bug in the tests themselves. For the most part, I'm inclined to agree. However, there are scenarios where having each test build the required state from scratch is prohibitively expensive. Imagine if you worked at NASA wanted to run test_booster_ignition(), test_booster_cutoff(), test_second_stage_ignition(), and test_self_destruct(). I suppose you could run them in random order, but you'd use up a lot of rockets that way. Somewhat more seriously, let's say you wanted to do test queries against a database with 100 million records in it. You could rebuild the database from scratch for each test, but doing so might take hours per test. Sometimes, real life is just *so* inconvenient. There is another dependency and that I'd call a logical dependency. This occurs when e.g. test X tests for an API presence and test Y tests the API behaviour. In other words, Y has no chance to succeed if X already failed. Sure. I run into that all the time. A trivial example would be the project I'm working on now. I've come to realize that a long unbroken string of E's means, "Dummy, you forgot to bring the application server up before you ran the tests". It would be nicer if the test suite could have run a single test which proved it could create a TCP connection and when that failed, just stop. Many test cases in the Python test suite have multiple asserts. I believe both resource sharing and sequential dependencies are reasons. I consider 'one assert (test) per testcase' to be on a par with 'one class per file'. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
In article <11jrt8-l32@satorlaser.homedns.org>, Ulrich Eckhardt wrote: > > Some people advocate that the test framework should > > intentionally randomize the order, to flush out inter-test dependencies > > that the author didn't realize existed (or intend). > > If you now > happen to influence one test with another and the next run randomizes > the tests differently, you will never see the fault again. Without this > reproducability, you don't gain anything but the bad stomach feeling > that something is wrong. The standard solution to that is to print out the PRNG initialization state and provide a way in your test harness to re-initialize it to that state. I've done things like that in test scenarios where it is difficult or impossible to cover the problem space deterministically. > Your unfortunate case is where test X creates persistent state that must > be present in order for test X+1 to produce meaningful results. This > kind of dependency obviously blows, as it means you can't debug test X+1 > separately. I'd call this operational dependency. > > This kind of dependency is IMHO a bug in the tests themselves. For the most part, I'm inclined to agree. However, there are scenarios where having each test build the required state from scratch is prohibitively expensive. Imagine if you worked at NASA wanted to run test_booster_ignition(), test_booster_cutoff(), test_second_stage_ignition(), and test_self_destruct(). I suppose you could run them in random order, but you'd use up a lot of rockets that way. Somewhat more seriously, let's say you wanted to do test queries against a database with 100 million records in it. You could rebuild the database from scratch for each test, but doing so might take hours per test. Sometimes, real life is just *so* inconvenient. > There is another dependency and that I'd call a logical dependency. This > occurs when e.g. test X tests for an API presence and test Y tests the > API behaviour. In other words, Y has no chance to succeed if X already > failed. Sure. I run into that all the time. A trivial example would be the project I'm working on now. I've come to realize that a long unbroken string of E's means, "Dummy, you forgot to bring the application server up before you ran the tests". It would be nicer if the test suite could have run a single test which proved it could create a TCP connection and when that failed, just stop. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Mon, Jan 9, 2012 at 9:59 AM, Ulrich Eckhardt wrote: > There is another dependency and that I'd call a logical dependency. This > occurs when e.g. test X tests for an API presence and test Y tests the API > behaviour. In other words, Y has no chance to succeed if X already failed. > Unfortunately, there is no way to express this relation, there is no > "@unittest.depends(test_X)" to decorate test_Y with (Not yet!). Each test > would be the root of a tree of tests that it depends on. If a dependency > fails already, you can either skip the tree or at least mark the following > test failures as implicit failures, so that you can easily distinguish them > from the root cause. I can see where that could be useful. On the other hand, if such a decorator were included in unittest, I can already envision people abusing it to explicitly enshrine their operational dependencies, maybe even taking it as encouragement to write their tests in that fashion. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Am 09.01.2012 15:35, schrieb Roy Smith: The classic unittest philosophy says that tests should be independent of each other, which means they should be able to be run in arbitrary order. Some people advocate that the test framework should intentionally randomize the order, to flush out inter-test dependencies that the author didn't realize existed (or intend). While I agree with the idea, I'd like to add that independence is an illusion. You already have possible dependencies if you run tests in the same process/OS-installation/computer/parallel universe. If you now happen to influence one test with another and the next run randomizes the tests differently, you will never see the fault again. Without this reproducability, you don't gain anything but the bad stomach feeling that something is wrong. Test independence is a good thing. Many people don't understand this when writing tests, and inadvertently write tests that depend on each other. I've worked with those systems. They're a bear to debug. You've got some test suite that runs for 15 minutes and dies at case 37 of 43. If you try to run case 37 by itself, it won't run, and you can't figure out what state cases 1-36 were supposed to leave the system in to make 37 work. You could sink days or weeks into debugging this kind of crap. BTDT. I'm sorry to hear that, debugging other peoples' feces is never a task to wish for. That said, there are two kinds of dependencies and in at least this discussion there hasn't been any mentioning of the differences yet, but those differences are important. Your unfortunate case is where test X creates persistent state that must be present in order for test X+1 to produce meaningful results. This kind of dependency obviously blows, as it means you can't debug test X+1 separately. I'd call this operational dependency. This kind of dependency is IMHO a bug in the tests themselves. The unit testing framework could help you find those bugs by allowing random order of execution for the test cases. There is another dependency and that I'd call a logical dependency. This occurs when e.g. test X tests for an API presence and test Y tests the API behaviour. In other words, Y has no chance to succeed if X already failed. Unfortunately, there is no way to express this relation, there is no "@unittest.depends(test_X)" to decorate test_Y with (Not yet!). Each test would be the root of a tree of tests that it depends on. If a dependency fails already, you can either skip the tree or at least mark the following test failures as implicit failures, so that you can easily distinguish them from the root cause. This kind of dependency is quite normal although not directly supported by the unittest module. As a workaround, being able to define an order allows you to move the dependencies further to the top, so they are tested first. To sum it up, in order to catch operational dependencies, you need a random order while in order to clearly express logical dependencies in the output, you want a fixed order. Neither is the one true way, though my gut feeling is that the fixed order is overall more useful. As long as they understand the consequences of their actions, don't try to preach unittest religion to them. They're in the best position to know if what they're trying to do is the best thing for their particular situation. Amen! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 2012-01-09, Roy Smith wrote: > If somebody (i.e. the classic "consenting adult" of the Python > world) wants to take advantage of that to write a test suite > where the tests *do* depend on each other, and have to be run > in a certain order, there's nothing wrong with that. As long > as they understand the consequences of their actions, don't try > to preach unittest religion to them. They're in the best > position to know if what they're trying to do is the best thing > for their particular situation. If a question springs from an idea that is usually a bad practice, then it should be challenged. The possible broken-nose of a questioner is a small price to pay for the education of the peanut gallery. If a questioner does not wish to defend what they are doing, he or she has that right, of course. -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
In article , Ian Kelly wrote: > Randomizing the order is not a bad idea, but you also need to be able > to run the tests in a consistent order, from a specific random seed. > In the real world, test conflicts and dependencies do happen, and if > we observe a failure, make a change, rerun the tests and observe > success, we need to be able to be sure that we actually fixed the bug, > and that it didn't pass only because it was run in a different order. I've seen this argument play out multiple times on this group. Executive summary: OP: "I want to do X" Peanut Gallery: "You're not supposed to do that" Here's my commentary on that. The classic unittest philosophy says that tests should be independent of each other, which means they should be able to be run in arbitrary order. Some people advocate that the test framework should intentionally randomize the order, to flush out inter-test dependencies that the author didn't realize existed (or intend). Test independence is a good thing. Many people don't understand this when writing tests, and inadvertently write tests that depend on each other. I've worked with those systems. They're a bear to debug. You've got some test suite that runs for 15 minutes and dies at case 37 of 43. If you try to run case 37 by itself, it won't run, and you can't figure out what state cases 1-36 were supposed to leave the system in to make 37 work. You could sink days or weeks into debugging this kind of crap. BTDT. That being said, the unittest module, while designed to support the "all tests must be independent" philosophy, is a useful piece of software for lots of things. It provides an easy to use framework for writing tests, lots of convenient assertions, reporting, test discovery, etc, etc. If somebody (i.e. the classic "consenting adult" of the Python world) wants to take advantage of that to write a test suite where the tests *do* depend on each other, and have to be run in a certain order, there's nothing wrong with that. As long as they understand the consequences of their actions, don't try to preach unittest religion to them. They're in the best position to know if what they're trying to do is the best thing for their particular situation. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Am 09.01.2012 13:10, schrieb Lie Ryan: I was just suggesting that what the OP thinks he wants is quite likely not what he actually wants. Rest assured that the OP has a rather good idea of what he wants and why, the latter being something you don't know, because he never bothered to explain it and you never asked. Please don't think he's an idiot just because he wants something that doesn't make sense to you. *le sigh* Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/09/2012 09:03 AM, Eelco wrote: i havnt read every post in great detail, but it doesnt seem like your actual question has been answered, so ill give it a try. AFAIK, changing __dict__ to be an ordereddict is fundamentally impossible in python 2. __dict__ is a builtin language construct hardcoded into the C API. There is no way to mess with it. Apparently this is different in python 3, but I dont know much about that. Actually the primary question has been answered by Ian Kelly which suggested __prepare__ for Python 3, and Peter Otten posted a code for a custom TestLoader that will essentially do what the OP wanted. I was just suggesting that what the OP thinks he wants is quite likely not what he actually wants. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Jan 7, 2:06 am, Ian Kelly wrote: > wrote: > > Nonetheless, I'm still wondering if I could somehow replace the dict with an > > OrderedDict. > > In Python 3, yes. This is pretty much the entire use case for the new > __prepare__ method of metaclasses. See the "OrderedClass" example[...] This isn't accurate. The OrderedClass example uses an OrderedDict to remember the method creation order: def __new__(cls, name, bases, classdict): result = type.__new__(cls, name, bases, dict(classdict)) result.members = tuple(classdict) return result The instantiated objects __dict__ will still be a regularly dictionary, while the assignment order is stored in the class attribute .members. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
i havnt read every post in great detail, but it doesnt seem like your actual question has been answered, so ill give it a try. AFAIK, changing __dict__ to be an ordereddict is fundamentally impossible in python 2. __dict__ is a builtin language construct hardcoded into the C API. There is no way to mess with it. Apparently this is different in python 3, but I dont know much about that. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Fri, Jan 6, 2012 at 5:49 PM, Steven D'Aprano wrote: > In the real world, test conflicts and dependencies are bugs in your test > suite that should be fixed, like any other bug in code. The fact that it > is test code that is failing is irrelevant. I agree 100%, but none of that changes my point, which is that when that this sort of problem arises, you need to be able to test consistently to know that the bug is actually fixed. > Every test should stand alone. You should be able to isolate each and > every test and run it without the others. Ideally, yes. I'm talking about *unintentional* conflicts and dependencies. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/07/2012 11:49 AM, Steven D'Aprano wrote: You may not be able to run tests*simultaneously*, due to clashes involving external resources, but you should be able to run them in random order. tests that involves external resources should be mocked, although there are always a few external resources that cannot be mocked, those are the exceptions not the rule; a concurrent test runner might have mechanisms to mark them to be run synchronously. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Fri, 06 Jan 2012 10:20:28 -0700, Ian Kelly wrote: > On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan wrote: >> That unittest executes its tests in alphabetical order is >> implementation detail for a very good reason, and good unittest >> practice dictates that execution order should never be defined (some >> even argued that the execution order should be randomized). If the test >> runner turns out to execute tests concurrently, that should not cause >> problems for a well-designed test. Displaying the test results in a >> more convenient order for viewing is what you really wanted in 99.99% >> of the cases. > > Randomizing the order is not a bad idea, but you also need to be able to > run the tests in a consistent order, from a specific random seed. In the > real world, test conflicts and dependencies do happen, In the real world, test conflicts and dependencies are bugs in your test suite that should be fixed, like any other bug in code. The fact that it is test code that is failing is irrelevant. Also in the real world, sometimes it is too hard to fix a bug and you just live with it. > and if we observe > a failure, make a change, rerun the tests and observe success, we need > to be able to be sure that we actually fixed the bug, and that it didn't > pass only because it was run in a different order. Every test should stand alone. You should be able to isolate each and every test and run it without the others. If you can't, your test suite is buggy. (Chances are good that most test suites are buggy. After all, who writes tests for their tests? That's just the start of an infinite regress with rapidly diminishing benefit.) You may not be able to run tests *simultaneously*, due to clashes involving external resources, but you should be able to run them in random order. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Sat, 07 Jan 2012 04:01:44 +1100, Lie Ryan wrote: > On 01/07/2012 12:36 AM, Ulrich Eckhardt wrote: >> True, perhaps, but doing it this way would be more fun and easier >> reusable in other cases where the default order is not desirable. I can >> also go and name the test functions test_000 to test_009 to get results >> quickly, if that was the only goal. > > Fun and easier, perhaps. Except that it solves the wrong problem. Fun and easier, and a terrible thing to do. Contrast: "test_binsearch_tuple is failing. What does that mean?" "Oh, that tests that binsearch works on a tuple. You need to look at the BinSearch.search_tuple method and see what it's doing." with "test_047 is failing. What does that mean?" "How the hell should I know?" Debugging is hard enough without obfuscating the test names. You wouldn't write a function called "func_009", why write one called "test_009"? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 6 January 2012 13:40, Ulrich Eckhardt wrote: > Am 06.01.2012 12:44, schrieb Peter Otten: > >> Alternatively you can write your own test loader: > > [...CODE...] > > Well, actually you just did that for me and it works! ;) > > > Nonetheless, I'm still wondering if I could somehow replace the dict with an > OrderedDict. No, you can't. -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 6 January 2012 11:44, Peter Otten <__pete...@web.de> wrote: > class Loader(unittest.TestLoader): > def getTestCaseNames(self, testCaseClass): > """Return a sequence of method names found within testCaseClass > sorted by co_firstlineno. > """ That's a clever trick! -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/07/2012 04:20 AM, Ian Kelly wrote: On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan wrote: That unittest executes its tests in alphabetical order is implementation detail for a very good reason, and good unittest practice dictates that execution order should never be defined (some even argued that the execution order should be randomized). If the test runner turns out to execute tests concurrently, that should not cause problems for a well-designed test. Displaying the test results in a more convenient order for viewing is what you really wanted in 99.99% of the cases. Randomizing the order is not a bad idea, but you also need to be able to run the tests in a consistent order, from a specific random seed. In the real world, test conflicts and dependencies do happen, and if we observe a failure, make a change, rerun the tests and observe success, we need to be able to be sure that we actually fixed the bug, and that it didn't pass only because it was run in a different order. Concurrent testing is a bad idea for this reason -- it's not repeatable (testing concurrency, OTOH, is a perfectly fine thing to be thinking about). Concurrent testing is perfectly fine strategy in the case where you have thousands of tests and running them synchronously will just take too long. Certainly it makes it harder to repeat the test if there is any sort of dependency in the tests, but when you have the large number of tests, the benefit may exceeds the drawbacks. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan wrote: > That unittest executes its tests in alphabetical order is implementation > detail for a very good reason, and good unittest practice dictates that > execution order should never be defined (some even argued that the execution > order should be randomized). If the test runner turns out to execute tests > concurrently, that should not cause problems for a well-designed test. > Displaying the test results in a more convenient order for viewing is what > you really wanted in 99.99% of the cases. Randomizing the order is not a bad idea, but you also need to be able to run the tests in a consistent order, from a specific random seed. In the real world, test conflicts and dependencies do happen, and if we observe a failure, make a change, rerun the tests and observe success, we need to be able to be sure that we actually fixed the bug, and that it didn't pass only because it was run in a different order. Concurrent testing is a bad idea for this reason -- it's not repeatable (testing concurrency, OTOH, is a perfectly fine thing to be thinking about). Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/07/2012 12:36 AM, Ulrich Eckhardt wrote: Am 06.01.2012 12:43, schrieb Lie Ryan: On 01/06/2012 08:48 PM, Ulrich Eckhardt wrote: Hi! The topic explains pretty much what I'm trying to do under Python 2.7[1]. The reason for this is that I want dir(SomeType) to show the attributes in the order of their declaration. This in turn should hopefully make unittest execute my tests in the order of their declaration[2], so that the output becomes more readable and structured, just as my test code (hopefully) is. IMO that's a futile effort, first, because as you already know, the test should not rely on the order. If you want the result to be printed in a certain order, that's a display issue, not testing order issue. I'm not sure if you just -ahem- enjoy Usenet discussions, but please read the footnote that explains that I don't want to discuss this part and why you are actually wrong with your partial understanding of the issue. ;^) I fully understand your issue, and I stand by my opinion. I believe you're misunderstanding the nature of your problem, your issue is not that you want a customized test order execution, but you want a customized view of the test result. That unittest executes its tests in alphabetical order is implementation detail for a very good reason, and good unittest practice dictates that execution order should never be defined (some even argued that the execution order should be randomized). If the test runner turns out to execute tests concurrently, that should not cause problems for a well-designed test. Displaying the test results in a more convenient order for viewing is what you really wanted in 99.99% of the cases. > I guess you would have better luck doing what you want by customizing > the TestResult or TestRunner. True, perhaps, but doing it this way would be more fun and easier reusable in other cases where the default order is not desirable. I can also go and name the test functions test_000 to test_009 to get results quickly, if that was the only goal. Fun and easier, perhaps. Except that it solves the wrong problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Fri, Jan 6, 2012 at 9:06 AM, Ian Kelly wrote: > On Fri, Jan 6, 2012 at 6:40 AM, Ulrich Eckhardt > wrote: >> Nonetheless, I'm still wondering if I could somehow replace the dict with an >> OrderedDict. > > In Python 3, yes. This is pretty much the entire use case for the new > __prepare__ method of metaclasses. See the "OrderedClass" example at: > http://docs.python.org/py3k/reference/datamodel.html#special-method-names That link should be: http://docs.python.org/py3k/reference/datamodel.html#customizing-class-creation -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On Fri, Jan 6, 2012 at 6:40 AM, Ulrich Eckhardt wrote: > Nonetheless, I'm still wondering if I could somehow replace the dict with an > OrderedDict. In Python 3, yes. This is pretty much the entire use case for the new __prepare__ method of metaclasses. See the "OrderedClass" example at: http://docs.python.org/py3k/reference/datamodel.html#special-method-names Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Ulrich Eckhardt wrote: Hi! The topic explains pretty much what I'm trying to do under Python 2.7[1]. The reason for this is that I want dir(SomeType) to show the attributes in the order of their declaration. This in turn should hopefully make unittest execute my tests in the order of their declaration[2], so that the output becomes more readable and structured, just as my test code (hopefully) is. I believe unittest executes tests in alphabetical order, but it does not display them in the same order when it prints error/fail results. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Am 06.01.2012 12:44, schrieb Peter Otten: Alternatively you can write your own test loader: [...CODE...] Well, actually you just did that for me and it works! ;) Nonetheless, I'm still wondering if I could somehow replace the dict with an OrderedDict. Thank you! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Am 06.01.2012 12:43, schrieb Lie Ryan: On 01/06/2012 08:48 PM, Ulrich Eckhardt wrote: Hi! The topic explains pretty much what I'm trying to do under Python 2.7[1]. The reason for this is that I want dir(SomeType) to show the attributes in the order of their declaration. This in turn should hopefully make unittest execute my tests in the order of their declaration[2], so that the output becomes more readable and structured, just as my test code (hopefully) is. IMO that's a futile effort, first, because as you already know, the test should not rely on the order. If you want the result to be printed in a certain order, that's a display issue, not testing order issue. I'm not sure if you just -ahem- enjoy Usenet discussions, but please read the footnote that explains that I don't want to discuss this part and why you are actually wrong with your partial understanding of the issue. ;^) > I guess you would have better luck doing what you want by customizing > the TestResult or TestRunner. True, perhaps, but doing it this way would be more fun and easier reusable in other cases where the default order is not desirable. I can also go and name the test functions test_000 to test_009 to get results quickly, if that was the only goal. Cheers! Uli -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
Ulrich Eckhardt wrote: > The topic explains pretty much what I'm trying to do under Python > 2.7[1]. The reason for this is that I want dir(SomeType) to show the > attributes in the order of their declaration. This in turn should > hopefully make unittest execute my tests in the order of their > declaration[2], so that the output becomes more readable and structured, > just as my test code (hopefully) is. > > I've been toying around with metaclasses, trying to replace __dict__ > directly, or using type() to construct the class, but either I hit > read-only attributes or the changes seem to be ignored. Alternatively you can write your own test loader: $ cat ordered_unittest2.py import unittest class Test(unittest.TestCase): def test_gamma(self): pass def test_beta(self): pass def test_alpha(self): pass class Loader(unittest.TestLoader): def getTestCaseNames(self, testCaseClass): """Return a sequence of method names found within testCaseClass sorted by co_firstlineno. """ def first_lineno(name): method = getattr(testCaseClass, name) return method.im_func.__code__.co_firstlineno function_names = super(Loader, self).getTestCaseNames(testCaseClass) function_names.sort(key=first_lineno) return function_names if __name__ == "__main__": unittest.main(testLoader=Loader()) $ python2.7 ordered_unittest2.py -v test_gamma (__main__.Test) ... ok test_beta (__main__.Test) ... ok test_alpha (__main__.Test) ... ok -- Ran 3 tests in 0.001s OK $ -- http://mail.python.org/mailman/listinfo/python-list
Re: replacing __dict__ with an OrderedDict
On 01/06/2012 08:48 PM, Ulrich Eckhardt wrote: Hi! The topic explains pretty much what I'm trying to do under Python 2.7[1]. The reason for this is that I want dir(SomeType) to show the attributes in the order of their declaration. This in turn should hopefully make unittest execute my tests in the order of their declaration[2], so that the output becomes more readable and structured, just as my test code (hopefully) is. IMO that's a futile effort, first, because as you already know, the test should not rely on the order. If you want the result to be printed in a certain order, that's a display issue, not testing order issue. I guess you would have better luck doing what you want by customizing the TestResult or TestRunner. -- http://mail.python.org/mailman/listinfo/python-list
replacing __dict__ with an OrderedDict
Hi! The topic explains pretty much what I'm trying to do under Python 2.7[1]. The reason for this is that I want dir(SomeType) to show the attributes in the order of their declaration. This in turn should hopefully make unittest execute my tests in the order of their declaration[2], so that the output becomes more readable and structured, just as my test code (hopefully) is. I've been toying around with metaclasses, trying to replace __dict__ directly, or using type() to construct the class, but either I hit read-only attributes or the changes seem to be ignored. As additional thought (but I'm not there yet), I guess that if I want dir(some_object) to be ordered similarly, I will also have to perform some steps on instance creation, not only on class creation, right? Thanks for any suggestions! Uli [1] I'm stuck with 2.x for now, from what I found it would be easier using 3.x. [2] No, don't tell me that the tests should run in any order. It's not the case that my tests depend on each other, but if one basic test for the presence of an API fails, the elaborate tests using that API will obviously fail, too, and I just want to take the first test that fails and analyse that instead of guessing the point to start debugging from the N failed tests. -- http://mail.python.org/mailman/listinfo/python-list
