Re: IRC discussion of smoking and branching
Hello, The discussion on smoking and branching is driven by the need to maintain quality across a wide range of compilers and target(1) machines. I would add a couple of points. 0. When working through the review process the tool I wanted the most was a simple way to test cross-platform/compiler after I had passed the test-suite on my machine. I have both a Linux and a Darwin platform but that is a fraction of the list. Waiting for some other kind soul to test things out for you on other platforms was slow, and would have been impossible without the usual suspects in #parrot . Could patches in a very specific format be mailed to a build farm, where compile tests/smoke tests would run against the HEAD + patches in the mail ? The reply could contain the test results , and even some sort of stamp, if it passed, that I could attach to the RT. With the build/smoke farm merit badge the reviewer of the changes can look at the meat of the changes without robotic and mind-numbing repetition of the test-suite. The time could be spent better considering the merits of the changes. 1. branching and merging in SVN is very primitive. tracking the last common ancestor must be done in the changelogs, failure to do so is a disaster. * This is an existing merge tool that matches up with the parallel development requirements: http://www.orcaware.com/svn/wiki/Svnmerge.py * This seems to be the official subversion page on merge-tracking http://subversion.tigris.org/merge-tracking/ * cherry picking never seems to work quite as well as you would want. The weak link is diff/patch. Unless the patches are extremely fine-grained you many times end up hand-editing hunks or doing post patch merges. I have tried subversion,git, hand-editing hunks (diff mode), and ediff. Only the last patch in a sequence of dependant changes can be modified without causing a tidal wave (good analogy http://en.wikipedia.org/wiki/Tsunami) of rejects. The end result is changes are not broken up, merger falls back on pass/reject. Are you ready for cherry picking ? 1. Is every whitespace cleanup, bug fix, refactor, and feature change a separate commit ? 2. Are your feature changes layered in even more separate commits ? each one compile/test-suite checked ? A. Why not wait until SVN has proper support for merging ? B. definitely need a tool of some sort to ensure the last common ancestor information is not mistakenly dropped. C. If there is a branch that is forked why not make it the stable branch ? The HEAD assuming the parrot project is flying along like we would all want it to is going to/does have heavy commit traffic on it. The number of changes you want in the stable is low. Put the expensive review checking on the merge from head - to stable, so the work-set can be reduced by a quick scan of the changelog to cherry pick which things could even be considered canidates. my 2 cents on the branching issue. I have been developing for myself tools to enhance svn for cherry picking, branching etc. I have working code; but that project is on the back-burner since diff/patch is the real problem, cherry picking/merging are the symptoms. All RCS systems use the patch/diff hunk as the fundamental change mechanism. There is just to much information dropped in a diff to handle conflicts cleanly, or rather the wrong kind of information to resolve conflicts in source code as well as it could. With that said I think the build/smoke farm would give developers a tool to widely test their changes, without expecting them to have a room full of computers. The branching model could be used, but it must be used in a way that is not going to push the current tools beyond their working use-cases, if that happens it will take alot of manual labor to compensate. Cheers, Mike Mattie (1) Autoconf lingo for the machine the program runs on. signature.asc Description: PGP signature
Re: IRC discussion of smoking and branching
On 3/29/07, Joshua Isom [EMAIL PROTECTED] wrote: On Mar 29, 2007, at 4:20 PM, jerry gay wrote: On 3/29/07, Nicholas Clark [EMAIL PROTECTED] wrote: particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? because not all coding standard tests are run with 'make test'. some tests were still under development, or too noisy, or took too long. this means that running make test and running prove t/codingstd will give you *very* different results. the correctness and performance problems with coding standard tests have largely been solved, and i'm now in favor of enabling all these tests during make test. this will require a large number of commits up front to fix the current list of failures, but would prevent developers from committing code that's not up to snuff. ~jerry Should we even require all of these tests to be ran by default? These tests should never fail for a user compiling a release version of parrot, so should they need to test them? They're good for developers, but only developers. And from a prove t/codingstd, should parrot's tests test any of languages? A language's coding standards should be a personal preference of the language author. Plus, some of the failures are from the tests testing documentation as well as source, so it seems to be slightly exaggerated. until we stop actively developing major subsystems we should be running these tests regularly. no test should fail for a user compiling a release version of parrot. should we run any tests at all? of course. we exclude languages from coding standard tests as a general rule. if some tests are testing languages/ files, either the test is broken, or the authors requested it (e.g. t/codingstd/perlcritic.t.) there have been tickets entered for this in the past, and just yesterday i cleaned up two test files that were testing languages--work in this area continues. this is part of the correctness measure i mentioned has largely been solved. please enter tickets for anything that looks wrong, we'll get it fixed up. our documentation has standards, too. for example, every op should have a doc; no trailing spaces; line length, etc. i can't see a reason not to test documentation. ~jerry
Re: IRC discussion of smoking and branching
jerry gay wrote: Should we even require all of these tests to be ran by default? These tests should never fail for a user compiling a release version of parrot, so should they need to test them? They're good for developers, but only developers. until we stop actively developing major subsystems we should be running these tests regularly. no test should fail for a user compiling a release version of parrot. should we run any tests at all? of course. I concur that the user shouldn't get failing tests for things like whitespace at the end of lines. More importantly, the user shouldn't be wasting time running tests for coding standards and documentation. How about a 'make devtest' target that runs standards/doc tests first, and then the ordinary 'make test' target? It would require all developers to get into the habit of running 'make devtest' instead of 'make test', but that's not a heavy burden. Allison
Re: IRC discussion of smoking and branching
Will Coleda wrote: So lets document what we need. Right now 'make smoke' generates an HTML report which is uploaded to the smoke server. Talk has happened in the past about making this more DB like instead of rendered output, but my concern is for the user visible features we're lacking. Perhaps: 1) Notify mailing list on failure (threshold of 9x%?) 2) Notify mailing list on new platform/compiler? 3) Better GUI on the server, including: a - sort by foo? b - highlight platforms with failures on summary screen? c - ability to which tests are failing across platforms. only on single platform? All good, with (1) being the first priority. Once we've decided what we want/need, we can open a ticket (or tickets on various components) and worry about implementation. Yes, please proceed. And I suggest looking at Mozilla's Tinderbox, when we get to thinking about implementation. Allison
Re: IRC discussion of smoking and branching
On Fri, Mar 30, 2007 at 08:58:19AM -0700, Allison Randal wrote: I concur that the user shouldn't get failing tests for things like whitespace at the end of lines. More importantly, the user shouldn't be wasting time running tests for coding standards and documentation. How about a 'make devtest' target that runs standards/doc tests first, and then the ordinary 'make test' target? It would require all developers to get into the habit of running 'make devtest' instead of 'make test', but that's not a heavy burden. An alternative is to have Cmake test be an alias, either to Cmake devtest by default, and a smaller Cmake usertest (or somesuch) when the source tree is an official release. Having the source tree know when it's an official release (perhaps by including or not including a file) would also allow the parrot executable to report accurately whether it's the official release, or one patched in some local way. We've not figured out a good way to do this for Perl 5 yet. For Perl I'd really like it if even perl -v (the short version output) said I'm patched somewhere, so that it was almost impossible to submit a bug report without that piece of information being forthcoming. (You don't need to know what the patches are immediately, because at least you now have enough information to ask the bug reporter to run perl -V or the equivalent, to actually find the information out) I assume that in the position of maintaining Parrot I'd like the same information about whether the release is vanilla and unmolested when I got a bug report. Nicholas Clark
Re: IRC discussion of smoking and branching
On Friday 30 March 2007 09:36, Nicholas Clark wrote: An alternative is to have Cmake test be an alias, either to Cmake devtest by default, and a smaller Cmake usertest (or somesuch) when the source tree is an official release. Having the source tree know when it's an official release (perhaps by including or not including a file) would also allow the parrot executable to report accurately whether it's the official release, or one patched in some local way. I like this idea. The easiest way not to make a mistake is if you don't have the option to make a mistake. For Perl I'd really like it if even perl -v (the short version output) said I'm patched somewhere, so that it was almost impossible to submit a bug report without that piece of information being forthcoming. (You don't need to know what the patches are immediately, because at least you now have enough information to ask the bug reporter to run perl -V or the equivalent, to actually find the information out) I assume that in the position of maintaining Parrot I'd like the same information about whether the release is vanilla and unmolested when I got a bug report. This one too. -- c
Re: IRC discussion of smoking and branching
On Tue, Mar 27, 2007 at 01:45:57PM -0700, Allison Randal wrote: Steve_p It works OK if everyone agrees that one ( or a very few) access the maintanence branch bernhard How many branches are we talking about 1,2 or 10 ? chromatic Steve_p, I think Nicholas might disagree that it works okay. Yes and no. I don't think that the stable/development spit in Perl 5 land is broken. There is a problem that there aren't enough people with good enough knowledge to be committers, and in particular to want to review and apply patches supplied by non-committers, which are often for areas that no-one is totally comfortable with. [Also, there are patches that turn out to be works for me whereas really for something as widespread and production critical as Perl 5 you want I understand this code, and this is the right way to solve it patches] Slightly detached from this is that very few people want to diagnose or fix bugs. (Which is a job, rather than fun, unless your sense of fun is somewhat puritanical*). You don't need commit access for this. particle allison: as long as the development branches are kept up-to-date wrt the stable branch, they won't diverge pmichaud well, if we treat the trunk branch as being the stable one, then there's a built-in incentive to get things checked into the trunk because without that new features don't make it into the release chromatic particle, that's never been my experience. pmichaud s/checked in/merged in/ allison particle: it depends on someone actively watching and merging changes, which is pretty much a full-time job for a volunteer allison (and not a particularly fun job) particle it's a job we all do already, in a way My estimate is that for Perl 5, keeping the maintenance branch maintained is about 1 day per week. I think Rafael said that it takes about 1 day per week to keep on top of incoming patches for Perl 5. So it is a lot for a volunteer. chromatic It takes the same kind of developer discipline to manage a long-lived branch as it does to keep the trunk stable, and managing branches adds administrative overhead and latency. In Perl 5, there is no 5.10 branch. It's the trunk. The principle reason to have a maintenance branch is that 5.8.$n+1 is binary compatible with 5.8.$n, which means that not all changes on the trunk are suitable for 5.8.x. Applying the Perl 5 philosophy to Parrot, there wouldn't be any need for a branch until 1.0 is released. chromatic When smokes fail, Paul Cochrane fixes a bunch of whitespace issues. Why do smokes fail? Are people committing things without running make test? If I arrange to kidnap Paul Cochrane, will it force a proper solution? :-) allison I'm totally on board for improving our smoke system Most Perl 5 smoke systems report the bad tidings of black smoke to perl5-porters. I've never noticed a failing Parrot smoke report to this list, so I infer that they aren't set up this way. Would changing that help focus minds? particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? Nicholas Clark * there's a better word than this, but it slips my mind. It's not Presbyterian, but it's a word I'd associate with it.
Re: IRC discussion of smoking and branching
On Thursday 29 March 2007 13:05, Nicholas Clark wrote: I don't think that the stable/development spit in Perl 5 land is broken. There is a problem that there aren't enough people with good enough knowledge to be committers, and in particular to want to review and apply patches supplied by non-committers, which are often for areas that no-one is totally comfortable with. [Also, there are patches that turn out to be works for me whereas really for something as widespread and production critical as Perl 5 you want I understand this code, and this is the right way to solve it patches] Slightly detached from this is that very few people want to diagnose or fix bugs. (Which is a job, rather than fun, unless your sense of fun is somewhat puritanical*). You don't need commit access for this. Okay, if not broken it's slightly bent. chromatic When smokes fail, Paul Cochrane fixes a bunch of whitespace issues. Why do smokes fail? Sometimes there are platform-specific problems. Are people committing things without running make test? Not me, but I can't speak for everyone. If I arrange to kidnap Paul Cochrane, will it force a proper solution? :-) More likely, we'll learn to live with more broken windows. allison I'm totally on board for improving our smoke system Most Perl 5 smoke systems report the bad tidings of black smoke to perl5-porters. I've never noticed a failing Parrot smoke report to this list, so I infer that they aren't set up this way. Would changing that help focus minds? That's the plan. particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? The line-ending coding standards tests can be a problem in some cases, where Windows developers add new files with their native format and forget to set the svn:eol-style=native property on the files, so those standards fail on Unix and Unix-like platforms. I honestly thought we had a little program that added files, updated the MANIFEST appropriately, and set the right properties, but I can't find it anymore. I know I've written that program for a handful of projects already anyway. As for the rest, I can't speculate. -- c
Re: IRC discussion of smoking and branching
On 3/29/07, Nicholas Clark [EMAIL PROTECTED] wrote: particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? because not all coding standard tests are run with 'make test'. some tests were still under development, or too noisy, or took too long. this means that running make test and running prove t/codingstd will give you *very* different results. the correctness and performance problems with coding standard tests have largely been solved, and i'm now in favor of enabling all these tests during make test. this will require a large number of commits up front to fix the current list of failures, but would prevent developers from committing code that's not up to snuff. ~jerry
Re: IRC discussion of smoking and branching
particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? The line-ending coding standards tests can be a problem in some cases, where Windows developers add new files with their native format and forget to set the svn:eol-style=native property on the files, so those standards fail on Unix and Unix-like platforms. Hopefully this problem won't turn up so often as I've added t/distro/file_metadata.t to the default list of tests for 'make test' . This does depend on people using 'make test' before committing (and as much as I feel I'm helping out with fixing whitespace issues, I still wish I could fix other broken windows) Paul
Re: IRC discussion of smoking and branching
chromatic == chromatic [EMAIL PROTECTED] writes: chromatic The line-ending coding standards tests can be a problem chromatic in some cases, where Windows developers add new files chromatic with their native format and forget to set the chromatic svn:eol-style=native property on the files, so those chromatic standards fail on Unix and Unix-like platforms. If it helps any: it's possible to fiddle with one's ~/.subversion/config file so that it automatically puts that property (or any property, really) on newly-added files whose names match a pattern. This isn't a foolproof solution, because it requires that every committer make this change on every box on which they hack, and I don't know if there's an equivalent mechanism for those who use git or svk. But it could be a start. Here's the relevant bits from my config file: [miscellany] ### Set enable-auto-props to 'yes' to enable automatic properties ### for 'svn add' and 'svn import', it defaults to 'no'. ### Automatic properties are defined in the section 'auto-props'. enable-auto-props = yes [auto-props] *.pl = svn:eol-style=native;svn:executable;svn:keywords=Id *.pm = svn:eol-style=native;svn:keywords=Id -- Покажи мне твой .emacs, и я скажу, кто ты. -- Russian Proverb
Re: IRC discussion of smoking and branching
Eric Hanchrow wrote: Here's the relevant bits from my config file: [miscellany] ### Set enable-auto-props to 'yes' to enable automatic properties ### for 'svn add' and 'svn import', it defaults to 'no'. ### Automatic properties are defined in the section 'auto-props'. enable-auto-props = yes [auto-props] *.pl = svn:eol-style=native;svn:executable;svn:keywords=Id *.pm = svn:eol-style=native;svn:keywords=Id To which you might want to add: *.t = svn:eol-style=native;svn:keywords=Id;svn:mime-type=text/plain
Re: IRC discussion of smoking and branching
Eric Hanchrow wrote: Here's the relevant bits from my config file: [miscellany] ### Set enable-auto-props to 'yes' to enable automatic properties ### for 'svn add' and 'svn import', it defaults to 'no'. ### Automatic properties are defined in the section 'auto-props'. enable-auto-props = yes [auto-props] *.pl = svn:eol-style=native;svn:executable;svn:keywords=Id *.pm = svn:eol-style=native;svn:keywords=Id To which you might want to add: *.t = svn:eol-style=native;svn:keywords=Id;svn:mime-type=text/plain
Re: IRC discussion of smoking and branching
On Mar 27, 2007, at 4:45 PM, Allison Randal wrote: particlebut, we need better smoke tools So lets document what we need. Right now 'make smoke' generates an HTML report which is uploaded to the smoke server. Talk has happened in the past about making this more DB like instead of rendered output, but my concern is for the user visible features we're lacking. Perhaps: 1) Notify mailing list on failure (threshold of 9x%?) 2) Notify mailing list on new platform/compiler? 3) Better GUI on the server, including: a - sort by foo? b - highlight platforms with failures on summary screen? c - ability to which tests are failing across platforms. only on single platform? Pugs has done some nifty stuff with their smokes since we originally borrowed the concept. Perhaps we can rebase to their current code. Once we've decided what we want/need, we can open a ticket (or tickets on various components) and worry about implementation. Feedback, please. -- Will Coke Coleda [EMAIL PROTECTED]
Re: IRC discussion of smoking and branching
On Mar 29, 2007, at 10:14 PM, Will Coleda wrote: On Mar 27, 2007, at 4:45 PM, Allison Randal wrote: particlebut, we need better smoke tools So lets document what we need. Right now 'make smoke' generates an HTML report which is uploaded to the smoke server. Talk has happened in the past about making this more DB like instead of rendered output, but my concern is for the user visible features we're lacking. Perhaps: 1) Notify mailing list on failure (threshold of 9x%?) 2) Notify mailing list on new platform/compiler? 3) Better GUI on the server, including: a - sort by foo? b - highlight platforms with failures on summary screen? c - ability to which tests are failing across platforms. only on single platform? 4) Not have to search through a whole page to find that one failing test, or unexpected succeeded test, etc. 5) Easily compare different revisions of one platform, and two platforms with near revisions, since unless we get cluster of very heterogeneous computers, we must rely on user submissions. Updates to languages/ don't really matter unless you're smoking languages for instance, and sometimes a lot of the revision updates are to languages/. Without a lot of work, I can't go to the smoke page and find out the failing differences between revision, say 16000 and 17000. Pugs has done some nifty stuff with their smokes since we originally borrowed the concept. Perhaps we can rebase to their current code. Once we've decided what we want/need, we can open a ticket (or tickets on various components) and worry about implementation. Feedback, please. One other thing I've noticed is that todo tests sometimes become forgotten tests. And since they're sometimes platform specific, they don't get fixed for that platform because feature x doesn't have the code support. Other than doing a grep of t/ or something similar, there's no simple way for being aware of bugs that are todo'd. -- Will Coke Coleda [EMAIL PROTECTED]
Re: IRC discussion of smoking and branching
On Thursday 29 March 2007 23:03, Joshua Isom wrote: One other thing I've noticed is that todo tests sometimes become forgotten tests. And since they're sometimes platform specific, they don't get fixed for that platform because feature x doesn't have the code support. Other than doing a grep of t/ or something similar, there's no simple way for being aware of bugs that are todo'd. I have a prototype harness that uses TAP::Parser to report these: http://www.oreillynet.com/onlamp/blog/2007/03/cpan_module_review_tapparser.html -- c
Re: IRC discussion of smoking and branching
On Mar 29, 2007, at 4:20 PM, jerry gay wrote: On 3/29/07, Nicholas Clark [EMAIL PROTECTED] wrote: particle and i'm not interested in testing every revision, when so many might be coding standards Why are people even checking things in that fail coding standards? because not all coding standard tests are run with 'make test'. some tests were still under development, or too noisy, or took too long. this means that running make test and running prove t/codingstd will give you *very* different results. the correctness and performance problems with coding standard tests have largely been solved, and i'm now in favor of enabling all these tests during make test. this will require a large number of commits up front to fix the current list of failures, but would prevent developers from committing code that's not up to snuff. ~jerry Should we even require all of these tests to be ran by default? These tests should never fail for a user compiling a release version of parrot, so should they need to test them? They're good for developers, but only developers. And from a prove t/codingstd, should parrot's tests test any of languages? A language's coding standards should be a personal preference of the language author. Plus, some of the failures are from the tests testing documentation as well as source, so it seems to be slightly exaggerated.
IRC discussion of smoking and branching
A discussion of draft document http://rakudo.org/parrot/index.cgi?parallel_development_requirements, with some side threads edited out. See http://www.parrotcode.org/misc/parrotsketch-logs/irclog.parrotsketch-200703/irclog.parrotsketch.20070327 for the full text. ... Coke ok. This is basically a followup to discussions that have occurred on #parrot and pp - what are our expectations about trunk in terms of stability? how can we use branching and tools to support these expectations? Cokethe outline of the various topics is on the wiki. Coke(and I have a meeting in 8 minutes. =-) allison ok, let's continue the conversation on the list, then bernhardHave a bugfix branch for every release ? CokeThat's certainly a possibility, b. bernhardGives some freedom on trunk chromatic Branches suck though. CokeI would need you to specify the ways in which they suck. =-) chromatic Merging between branches sucks. chromatic Really sucks. Cokechromatic: thank you. particleparallel development sucks, just like multiple inheritance. chromatic Pulling code from experimental branches into a stable branch is a lot of work. particlebut we need it. chromatic It's a lot of not-fun work. Coke I think that svk or some other sits in front of svn method of keeping track of thigns would work. especially if those svk repos were public. chromatic It doesn't work very well in Perl 5, and I fear that it won't work very well anywhere else. particlewe need an svn-only solution Cokethen it's easy to keep things in sync. allison coke: I've had terrible trouble with bugs in svk messing up my repository, I've gone back to svn because it's reliable. Coke particle: I disagree. We need something that does svn in the back end, but I don't think we have to have the LCD be the solution everyone uses. Steve_p It works OK if everyone agrees that one ( or a very few) access the maintanence branch bernhardHow many branches are we talking about 1,2 or 10 ? chromatic Steve_p, I think Nicholas might disagree that it works okay. Coke alrightee. General plan: update the wiki page with your notes. Someone (probably me) will start a thread on p2p. Chromatic will herd you for the remainder of this meeting. =-) Coke any discussion that continues here, I or someone else will capture and add to the wiki. pmichaudthanks, Coke! particle my proposed solution would support infinite branches, infinite repositories. the main repository would have (at least) an integration branch, and a stable branch Steve_p Oh yes, and unfortunately, scm tools tend to mess up branching. The human factor of branching comes int too. tewk(chromatic's comments)++ CokeRoger. Hope everyone is having fun. (and let me know if you're not!) allison the most important question is what problem are we solving? allison once we have that, it'll make it much easier to pick the right solution particlewe're aiming to solve multiple problems particle~ provide a stable branch chromatic It sounds like the problem is Some changes are too large to keep stable in small patches. bernhardLessen fear of breaking things particle parallel work on multiple subsystems may result in diverging code. this is the hard part to manage pmichaud out of curiosity (because I don't know), how do other large systems like the Linux kernel handle it? jisom_ then we need a better test suite! :) jisom_ pmichaud, luck chromatic pmichaud, the Linux kernel has lots of public trees, from which various maintainers pull patchsets when they want to include them. chromatic It's highly distributed, so there's no single stable trunk. pmichaudchromatic: is that a function of maturity, then? chromatic More a function of chaos. pmichaudheh pmichaudokay :-) pmichaudanswers my question! particlepmichaud: linux kernel folk use 'git', a tool linus wrote particleit's not portable enough for us, yet chromatic They use git now, yes, but they used to mail patches around and there was no central SCM besides Linus' inbox. pmichaud particle: I meant more at a meta-level, in terms of managing multiple repositories and dealing with parallel work on multiple subsystems particle yes, it's a distributed vcs solution. we're used to, and i think aiming for, a centralized solution pmichaud I didn't follow some of the earlier discussions, but what are some examples of specific obstacles we're encountering with the present system? particlepresently, trunk breaks. particlefar too often.
