On Tue, Jun 26, 2012 at 6:37 PM, Aaron Meurer <asmeu...@gmail.com> wrote: > On Jun 26, 2012, at 6:27 PM, "Ondřej Čertík" <ondrej.cer...@gmail.com> wrote: > >> Hi Stefan, >> >> On Mon, Jun 25, 2012 at 1:33 AM, krastanov.ste...@gmail.com >> <krastanov.ste...@gmail.com> wrote: >>>> How will this scale? I will also have spare machines to dedicate to >>>> this, once I get my new laptop. >>>> >>>> I think the idea behind sympy-bot work is that the app-engine server >>>> would keep track of what needs to be tested and distribute the work so >>>> that there is no duplication. >>> >>> I may be wrong, but I think that Certik actually wants the duplication >>> so more architectures are tested. >> >> Right. >> >> Btw, my first name is Ondrej. :) >> >>> >>> And if we actually do not want duplication, the easiest solution is >>> just to leave one single computer doing all the tests - the PR number >>> is low enough for one computer to handle them all even if it is >>> testing numerous configurations. I do not think it is worth it to make >>> sympy-bot any smarter. >> >> If Travis can do everything that we need, I am certainly ok to only use >> Travis. >> They will eventually update the Python to support "-R", so that's not >> a big deal. >> Based on our discussions, it seems that we really don't need to test >> various platforms >> as long as we use randomized hashing. >> >> My intuition tells me that there will be occasions to use sympy-bot, for >> example >> right now the "-R" testing, after Travis supports it, maybe some >> customized testing. >> It'd be nice if most of the complexity can be managed by the web app, and >> then >> the sympy-bot can stay "simple stupid". And I don't think it's that much >> work, >> but I don't have time for it at the moment. So I'll leave it as it is for >> now, >> I am personally very happy with Travis, and we'll see how it goes. >> >> >> However, I have a big problem with the randomized hashes. >> Our current status is that we can only pass tests without randomizing the >> hash. >> As such, at the moment, this is the only way that we can tell whether >> some pull request breaks something or not. >> So before we fix sympy, we should keep our main test runner >> (Travis) with a definite seed for the hashes. >> >> At the moment, your buildbot is running randomized hashes and so it >> almost always fails. >> This is useful for fixing sympy to actually work with randomized hashes, but >> the failures usually have nothing to do with the pull request at hand, >> and as such >> when working on a particular pull request (and not on fixing hash >> bugs), it's not very useful. >> >> So our main test running should not use random hashes, so that we can >> make sure that >> all tests actually pass on at least some configuration. >> >> Ondrej > > I disagree. One of the reasons that I pushed this pull request through > was so exactly this would happen. I'm hoping that it will get more > people to help out fixing these problems. Otherwise, it's just me who > really cares about them, and I don't have the resources to fix them > all in a reasonable time frame. I don't want to release unless the > tests pass in Python 3.3, and I do want to release, because it's been > almost a year again, and all the GSoC stuff from last summer is new > from the last release. > > I do understand that this makes it hard to tell if a pull request > breaks things or not. For now, the simplest solution is to rerun the > tests on master with the same hash seed. The better way is to send > pull requests fixing the errors.
I can see your point --- by making everybody to see the failures, you want to get more attention to it and get it fixed. Why not, let's get it fixed. I just don't want people's pull requests to get stalled by this, i.e. I want somehow to very robustly see if they break anything or not. The Travis tests do that currently, so that's an ok solution for me. Stefan, yes, please do run the tests as you do now. That is very valuable. Ondrej -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to sympy@googlegroups.com. To unsubscribe from this group, send email to sympy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
