As I noted on your pull request, I think a better way is to just wrap the output in an unordered type (set or dict), and let either the pretty printer or the doctest printer sort the output (both will do so). That way, we don't have to add any extra boilerplate to the doctests, which, as was noted, does make them harder to read as examples.
For the regular tests, from what I've seen so far, the problem is either sorted data types coming out in a different order (like lists from solve), or results coming out completely different (but still correct). The former case is easy to fix: just compare sets. For the latter, I'm not sure. A good example of this is cse(). You can get pretty different results. I looked at the code, and it's not only dependent on the order of .args, but also on an iteration through a dictionary (which is not guaranteed to be the same even without hash randomization). If someone wants to look at the cse() issue specifically, take a look at the failing test_issue_3022() from sympy/core/tests/test_expand.py. Aaron Meurer On Jun 30, 2012, at 5:13 AM, Sergiu Ivanov <unlimitedscol...@gmail.com> wrote: > Since I did take a bit of a look at doctest failures recently, I > thought I would make my conclusions public, although a lot of this is > going to be self-evident. > > Apparently, the reasons for doctest failures now belong to two groups: > > 1. instances of Basic are output, and some .args get output in > arbitrary order; > > 2. non-Basic thinks are output, and get output in arbitrary order. > > I suggest solving (1) by altering the corresponding printing functions > to sort their arguments, just as it has recently been done for > FiniteSet. This is an elementary change, it shouldn't impact the > performance of tests, since doctests don't generally get sufficiently > large for that. Nevertheless, I believe it is going to sort out a lot > of failures. > > Each failure of type (2) would require a more specific approach. > However, I'll try once again to push through the idea of defining > sympy.utilities.sympysort in the following way: > > sympysort(x) = sorted(x, key=default_sort_key) > > This will make the doctests which output non-Basic objects uglier, but > not less clearer with sympysort. Further, there are not that many > doctest failures of type (2), which makes me believe that resorting to > sympysort in a couple cases wouldn't be that fatal. > > I'm looking forward to suggestions. Whenever something more or less > final is decided, I'll implement it. > > Sergiu > > P.S. As noted in [0], altering the actual testing code is impractical > in solving such kind of failures. > > [0] https://github.com/sympy/sympy/pull/1393#issuecomment-6684465 > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To post to this group, send email to sympy@googlegroups.com. > To unsubscribe from this group, send email to > sympy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sympy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to sympy@googlegroups.com. To unsubscribe from this group, send email to sympy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
