On 6 Kvě, 12:23, Russell Keith-Magee <freakboy3...@gmail.com> wrote: > On Wed, May 6, 2009 at 7:53 AM, Henrique Romano <chrom...@gmail.com> wrote: > Like many things, it sounds like a good idea, but the devil is in the > details. A tool like this is really only useful if it provides triage > data that is both accurate and useful. Here's a quick checklist of the > basic questions a manual triager needs to answer: > > 1. Does the patch apply cleanly? > 2. Does the patch contain a test? > 3. Does the test fail if the rest of the patch isn't applied? > 4. Does it pass when the test is applied? > 5. Does the patch only solve the surface problem, or does an > underlying problem exist that needs fixing. > 6. Does the patch contain documentation?
Agreed, exactly was I was thinking about. > Now, some complications: > (1) is something that isn't that has an absolute answer - a patch may > apply to revision X, but may not apply cleanly to revision X+N. This > point needs to be re-established periodically. Agreed. Well I can think about having it fancier, but for basic I'd say it's enough to check periodically. (Pony thought: Database with what files are affected by which patch, scan timeline every night, check what files were affected by commits, re-run check for patches that affect those files too) > (2) requires inspection of the patch contents, but that bit isn't too > hard. However, there are conditions where not having a patch is > acceptable (things that aren't particularly testable, such as command > line behaviour), so the non-existence of a test doesn't necessarily > mean that a patch isn't ready for commit. IMO command line is also testable, althrough in an ugly way. Anyway; we should then create like "test required for this patch" trac checkbox, or just have "ready for bot" triage state only for more complicated and tests-requiring patches. > (3) and (4) require a buildbot to do automatically. I think about this tool as not a part of buildbot set and I don't think it require buildbot. This can be done on any machine with sanitized environment and do not require talking to buildbot. My idea is a bot that sits in the working directory of django trunk, applying patches for real, introspecting results by really running tests and svn st, and reporting results as oomments to the ticket directly in trac. > (5) requires human analysis. Oh I wish I could automate that ;) > (6) requires inspection of the patch. Again, depending on the ticket, > non-existence of docs may not be an problem, depending on the ticket. > > So - of the 6 tasks, only (1) can be done automatically, and even then > it needs to be updated regularly. (2), (5) and (6) require human > evaluation; (3) & (4) require a lot of infrastructure. AFAIK 3 and 4 require no infrastructure from Django itself and (2) and (6) could be easily determined if You will have a bot report (patch do not contain tests & docs and You can see from ticket desc it should be required). > You also need to consider the fact that Trac doesn't much automated > metadata to interpret which patches are valid at any given time. A > complex ticket may have multiple patches, only some of which will > apply at any given time; in addition, there may be competing > approaches being discussed, so multiple tickets may be valid. For start, I'd say applying latest diff should be enough. For future, I'd say we want to have special triage state for "ready for bot" and "bot approved" (easy with trac 0.11 custom workflow) and at this state, discussion should be resolved. I don't think it make so much sense for us to try patches that are not in their final shape. > That doesn't mean there isn't a place for a bot like you describe. > We're all in favor of any tool that will make the triage process > easier - however, you need to be clear about what the bot will > contribute to the process before you embark on building a complex > tool. Like I said, IMHO it can be made a bit easier by some "convention instead of configuration" decisions. But yes, still work it is. > Yours, > Russ Magee %-) Almad P.S.: I just learned Eric has some pieces of this functionality already and he's intrested in building some functionality during EDC sprints. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~----------~----~----~----~------~----~------~--~---
