On Sat, Dec 3, 2011 at 3:35 PM, Joachim Durchholz <j...@durchholz.org> wrote: > Am 03.12.2011 20:48, schrieb Aaron Meurer: > >> Right now, Stefan's script works only for his machine, if I were to >> run the same script, we would just get duplicate reports (unless we >> timed them to be offset or something). So we really need to get some >> poling mechanism implemented, > > > This sounds worrying. > > You see, what started as a simple buildbot now needs to coordinate stuff > across servers.
Not across servers, just between the app engine and multiple users (it's not much more than what already happens, if you look at the "coordination" graph). Sure, if multiple servers had to keep track of what has been reviewed, or is currently being reviewed, it would be a mess. But it doesn't seem too difficult for this (maybe I am missing some key thing?) > Which may fail, hang, or disconnect, complicating the image considerably. That's why you just have timeouts. If the local machine doesn't send back a report withint an hour (say), it will put it back in the queue. > And you also have some issues with identifying units of work, Maybe I'm misunderstanding what you mean by this, but isn't one unit of work always just a single test run? > making sure > each is run once Again, just use a priority queue. >, and avoid useless duplications (but you need to detect > whether a server failed because the test failed or because there was some > server mishap: in the former case, you wish to report the test as failed, in > the latter case, you need to schedule the test on some other machine). We basically already do this (albiet with some bugs, which are not hard to fix), because we want to avoid pushing up a failing test report that only failed because of some problem unrelated to the pull request. > You > may even want to supersede a server-failed test because the module change it > was supposed to test has been reverted or reworked. I don't follow what you are saying here. What do you mean by "supersede"? Do you mean to cancel the test, or something else? > > I'm a bit worried that we'll end working more on the testing framework than > on Sympy itself. It is a valid concern, I suppose. On the other hand, sympy-bot just works out of the box, and it has been working for some time. This is more than can be said about anything that I've seen so far with any Jenkins or otherwise. Personally, I prefer this approach because I don't really like (read, "don't know anything about") server hacking. I would have no idea how to setup a Jenkins server, but I have a somewhat good idea about how SymPy-bot works. Another thing to consider is that to modify Jenkins, you have to obtain ssh access to the machine (if you don't already have it), and go in and try to fix things, and hopefully you won't break anything. With SymPy-bot, it's just a git repo on GitHub. If you want to make a fix, you just submit a pull request. In other words, you get all the benefits of distributed version control (this actually isn't essential to the fact that it's a distributed testing service; is it possible to run the Jenkins info in a GitHub repo?). > I agree that UI quality is relevant. However, as I see lots of projects > using Hudson/Jenkins, I guess other factors have eclipsed any UI problems. I haven't spoken with many other people. Maybe it would be a good idea to do so. What are some projects that you know of that rely heavily on Jenkins? I guess we should talk to them and ask them how happy they are with it, and their thoughts on our idea here (btw, the polling and stuff is not really my idea; Ondrej came up with it and told me about it at the mentor summit). Because I know that sometimes these tools are used, but not actually like, but they are still used for whatever reason (there is no better alternative, it's too much work to switch, etc.). For example, at the mentor summit, Ondrej, Mateusz, and I went to a session about code reviews. The consensus I got from it was that different people have these different code review tools that they use, but no one is really happy with what they are using, except for the people using GitHub. > > This all doesn't mean that Jenkins is the way to go; I have too little > experience with automated testing to judge anything here. > I'm just worrying whether all relevant factors have been considered. Right. This is important. Though it should hopefully become clear if we start getting in over our heads. Aaron Meurer > > Regards, > Jo > > > -- > You received this message because you are subscribed to the Google Groups > "sympy" group. > To post to this group, send email to sympy@googlegroups.com. > To unsubscribe from this group, send email to > sympy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sympy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sympy" group. To post to this group, send email to sympy@googlegroups.com. To unsubscribe from this group, send email to sympy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sympy?hl=en.
