Re: rfc: safer smoking
On Wed, Jan 21, 2009 at 10:13:52AM +0800, imacat wrote: Oh... sorry. I do not know the cpan-testers-discuss list does not set the Reply-to: header, and I did not check it. Maybe I shall write to the list manager some other time. Here, have a procmail recipe to fix that :-) :0 fHw * ^List-Id:.*cpan-testers-discuss.perl.org | formail -i Reply-To: cpan-testers-discuss@perl.org -- David Cantrell | Nth greatest programmer in the world fdisk format reinstall, doo-dah, doo-dah; fdisk format reinstall, it's the Windows way
Re: rfc: safer smoking
On 01/20/2009 12:07 AM, David Westbrook wrote: On Sun, Jan 18, 2009 at 9:45 PM, imacat ima...@mail.imacat.idv.tw wrote: 2. Sorry I posted your mail on the list. I do not see any reason why this mail is off-list. Your first reply was directly to me, and not to the list (so the list is probably missing some context here), which is why i continued it off-list, assuming it was intentional. Oh... sorry. I do not know the cpan-testers-discuss list does not set the Reply-to: header, and I did not check it. Maybe I shall write to the list manager some other time. Proposition: + Provide an easy way to execute CPAN::Reporter::Smoker, limited just to distributions that have any version installed on the system (instead of all of CPAN). I still do not see at all. 1. How many smokers have installed my module? Only one, that is, I. How many smokers have installed Mac::iTune? No one. 2. With this new system you suggested, I will only get my own reports, which is nearly useless. And other authors will get no reports at all. 3. I will stop running smoke tests, since when I upgrade the modules weekly, the reports will be sent anyway. If I will not be sending reports other than the above, running smoke tests is a duplicated work and CPU waste which could be saved. In fact, currently the only modules I do not need to run smoke tests are those that I did not installed. 4. Nobody gets reports except for those popular modules like LWP, etc.. Also, nobody is running smoke tests anymore. 5. If nobody is running smoke tests, and nobody is receiving smoke test reports except LWP, how does this CPAN testers system work? So, why are you thinking that this will increase the amount of reports, but not demolish the whole CPAN testers system? If that is not what you mean, then I have a serious English problem. Either I misunderstand your proposition, or I misunderstand your expected result. In either way, I may have to restart my English class. -- Best regards, imacat ^_*' ima...@mail.imacat.idv.tw PGP Key: http://www.imacat.idv.tw/me/pgpkey.asc Woman's Voice News: http://www.wov.idv.tw/ Tavern IMACAT's: http://www.imacat.idv.tw/ TLUG List Manager: http://lists.linux.org.tw/cgi-bin/mailman/listinfo/tlug
Re: rfc: safer smoking
On Tue, Jan 20, 2009 at 9:13 PM, imacat ima...@mail.imacat.idv.tw wrote: Proposition: + Provide an easy way to execute CPAN::Reporter::Smoker, limited just to distributions that have any version installed on the system (instead of all of CPAN). I still do not see at all. 1. How many smokers have installed my module? Only one, that is, I. How many smokers have installed Mac::iTune? No one. For your module, and Mac::iTune, there won't be an added benefit. But there will be for more commonly installed modules. Which, of course, are already tested, but this could reach additional platforms/perls -- that potentially could be a bigger benefit than additional modules. 2. With this new system you suggested, I will only get my own reports, which is nearly useless. And other authors will get no reports at all. True, it's not reaching all of CPAN, but OTOH it's not zero modules either -- it's whatever's on your system .. which i'm guessing is 200-500 distros. 3. I will stop running smoke tests, since when I upgrade the modules weekly, the reports will be sent anyway. If I will not be sending reports other than the above, running smoke tests is a duplicated work and CPU waste which could be saved. Here's where are lines of thought split i think .. You wouldn't need to (and shouldn't) stop running smoke tests if your running this new system (C::R::S::Safer) .. There's two cases here: a) The system (read: platform/perl) you're smoking on is the same as where you're running C::R::S::Safer. + In this case, i'd say either don't run C::R::S::Safer, or make sure the reports-send.db file is shared between the C::R::S::Safer runs and the C::R::S smoke runs. That way there's no wasted resources. b) The platform/perl is different. This situation is what i'm targeting .. where you're smoking on system A, but also have system B, which full smoking is unsafe and/or inappropriate on. Running C::R::S::Safer on system B at least gets some partial smoke coverage/reports, whereas without it there wouldn't be any. Basically, it's not proposed to replace smoke testing -- just to be used where smoke testing can't (or shouldn't) be done. Your points 4. 5. and demolish the whole CPAN testers system followed from the assumption that running this would mean ceasing smoke testing, so i'll skip those. Take my actual setups for an example ... I have: * a $work laptop (winXP, 5.10) * a home desktop (win2K, 5.8.8) * a spare laptop (no hard drive) * hosting account (linux, i686, 5.8.8) * $work dev env (linux, 5.6.1) On the spare laptop, i'm booting a knoppix livecd and smoking from there (linux, i486, 5.8.8). BUT, the other 4 systems w/unique platforms/perls I can't run smoke testing on (for both safety and resources reasons). BUT, i am willing to run C::R::S::Safer, to ensure that i've contributed reports for everything that i have installed. So this is where I see it only creating additional testers reports. Basically, i envision the following levels of test report contribution: a) Install CPAN::Reporter, and send reports for any module that gets installed from that point on. b) Install CPAN::Reporter::Smoker::Safer, and run it to submit reports for all the distros you trust (read: have installed). And run it again in the future to test any newer versions of those distros, w/o actually installing them. c) Install run CPAN::Reporter::Smoker (or other smoker), and submit reports for everything. -- BUT, this takes an isolated dedicated environment. And it's key to note that these 3 are not mutually exclusive .. can do (c) on one system, and (a) and/or (b) on others (as in my setup described above). In either way, I may have to restart my English class. don't waste the time or money :) -- w/o your mention of it and the .tw email, I would have assumed the above was written by a native english speaker. This is a good thread -- parts of it will definitely make their way into the C::R::S::Safer documentation. --david
Re: rfc: safer smoking
On Wed, Jan 21, 2009 at 4:13 AM, imacat ima...@mail.imacat.idv.tw wrote: On 01/20/2009 12:07 AM, David Westbrook wrote: Proposition: + Provide an easy way to execute CPAN::Reporter::Smoker, limited just to distributions that have any version installed on the system (instead of all of CPAN). I still do not see at all. 1. How many smokers have installed my module? Only one, that is, I. How many smokers have installed Mac::iTune? No one. 2. With this new system you suggested, I will only get my own reports, which is nearly useless. And other authors will get no reports at all. 3. I will stop running smoke tests, since when I upgrade the modules weekly, the reports will be sent anyway. If I will not be sending reports other than the above, running smoke tests is a duplicated work and CPU waste which could be saved. In fact, currently the only modules I do not need to run smoke tests are those that I did not installed. 4. Nobody gets reports except for those popular modules like LWP, etc.. Also, nobody is running smoke tests anymore. 5. If nobody is running smoke tests, and nobody is receiving smoke test reports except LWP, how does this CPAN testers system work? So, why are you thinking that this will increase the amount of reports, but not demolish the whole CPAN testers system? This Safer CPAN Tester module could be installed by people like me who do don't do smoke testing at all (or by you but on your real development environment) thus it has the potential to increase the number of reports in more natural environments. One of my main issues with the tons of reports the main smokers send - despite the fact that I am totally thankful for them - that those reports are generated in a lab environment and their sheer number hides the few reports that come from real setups. Point: while I am happy to see my modules tested on all version of 5.8.x on NetBSD it would be more important to get the reports from people using the *default perl* of Red Hat 17.0 and Enterprise SuSE 38.7 or whatever version numbers they have. Those environments are usually more difficult and much more broken (!) than those of the smokers. This Safer smoker has the potential to help increasing this number. Gabor
Re: rfc: safer smoking
On Sun, Jan 18, 2009 at 9:45 PM, imacat ima...@mail.imacat.idv.tw wrote: 2. Sorry I posted your mail on the list. I do not see any reason why this mail is off-list. Your first reply was directly to me, and not to the list (so the list is probably missing some context here), which is why i continued it off-list, assuming it was intentional. The safer way is to stop the CPAN testers system. Can you qualify that? Isn't the CPAN testers system the whole reason we're here (this list, the cpantesters site, test reports, etc)?? The already-installed system may not meet the new requirement upon new releases. But that's fine .. and David Golden made a good point that this could be useful to check things before upgrading. The even-safer way is not to release anything. No software, no bugs, problems. ummm .. and no fun, and 0 chance of doing anything good. Isn't writing/releasing the whole point in any of us being programmers?? Yes, of course there's risk in any change/release, but the goal is that the benefits outweigh the risks. That aside, i'm not sure what not to release anything is in reference to here ... What is the right way? The right way is to solve the problem, not to close your eyes to the problems. I'm not sure i follow what problem is referring to in this context? I do not see your point. Is this because my English is too poor? English looks just fine to me (based on this and a couple previous posts on this list); but i'm a lowly monolingual and far from fluent in a 2nd language, so definitely not going to pass judgement. BUT, clearly we're on different wavelengths ... let's start back at the beginning: Proposition: + Provide an easy way to execute CPAN::Reporter::Smoker, limited just to distributions that have any version installed on the system (instead of all of CPAN). Pros: + Smoke testing benefits, in terms of generating reports (albeit not for all of CPAN). + Tests are submitted from actual use systems, as opposed to just an isolated builddir LIB path. + In theory is safer than full smoke, since only testing trusted distros (and their deps) -- the user trusted the distro enough to install it in the first place. + Potential use is to check things out before upgrading. + doesn't require (making the trust assumptions) a dedicated/isolated environment Cons: + In theory, still same potential risk as full smoke run + The trusted distros are based on a chain of assumptions. And then we can tackle each line item individually where we disagree. I do not see this also. If you remove all the reports from testers that are not really installing the module, the CPAN testers report database will be less than 1% left. I in no way was suggesting that ... just the opposite, i think this could be helpful to _increase_ the test count. e.g. i wouldn't have submitted a bunch of reports from my windows boxes w/o something like this. --david
Re: rfc: safer smoking
On Mon, Jan 19, 2009 at 6:07 PM, David Westbrook dwestbr...@gmail.com wrote: Proposition: + Provide an easy way to execute CPAN::Reporter::Smoker, limited just to distributions that have any version installed on the system (instead of all of CPAN). In general I think this is a good idea and I probably would run it on my development machine. Pros: + In theory is safer than full smoke, since only testing trusted distros (and their deps) -- the user trusted the distro enough to install it in the first place. For additional safety you might want to limit the installations to 1) Modules that already have N reports in the database meaning that some other testers have already tested it. 2) Modules that have been on CPAN for at least M days hoping that if the module and its tests do something bad someone has already caught this, reported to the rulers of CPAN and the module was taken off CPAN. regards Gabor
Re: rfc: safer smoking
--- Em seg, 19/1/09, Gabor Szabo szab...@gmail.com escreveu:
For additional safety you might want to limit the
installations to
1) Modules that already have N reports in the database
meaning that some other testers have already tested it.
2) Modules that have been on CPAN for at least M days
hoping that if the module and its tests do something
bad
someone has already caught this, reported to the
rulers of CPAN and the module was taken off
CPAN.
regards
Gabor
What about those distributions that have only that dummy test file with a
single BEGIN { use Foo::Bar; } that h2xs generates?
This actually does not means to much for me, but the smoke test (and mannually
too) will consider this distribution OK.
Shouldn't the smoke test include something like Devel::Cover?
Regards,
Alceu
Veja quais são os assuntos do momento no Yahoo! +Buscados
http://br.maisbuscados.yahoo.com
Re: rfc: safer smoking
On Mon, Jan 19, 2009 at 1:31 PM, Gabor Szabo szab...@gmail.com wrote: Pros: + In theory is safer than full smoke, since only testing trusted distros (and their deps) -- the user trusted the distro enough to install it in the first place. For additional safety you might want to limit the installations to 1) Modules that already have N reports in the database meaning that some other testers have already tested it. 2) Modules that have been on CPAN for at least M days hoping that if the module and its tests do something bad someone has already caught this, reported to the rulers of CPAN and the module was taken off CPAN. ah -- i like those! Suggestions on defaults for N M? I guess i'll look at my installed systems as examples and see what gets excluded for different values ... Any thoughts on CPAN::Reporter::Smoker::Safer for the namespace? i also like ::Installed, but taking into account the above, which shifts the focus a little, I'm leaning toward ::Safer. (does ::Safer imply a negative connotation towards CPAN::Reporter::Smoker?) thanks, --david
Re: rfc: safer smoking
On Mon, Jan 19, 2009 at 3:04 PM, David Westbrook dwestbr...@gmail.comwrote: (does ::Safer imply a negative connotation towards CPAN::Reporter::Smoker?) Not one that bothers me. After all, I put this in the C::R::S Pod: WARNING -- smoke testing is riskhttp://search.cpan.org/%7Edagolden/CPAN-Reporter-Smoker-0.17/lib/CPAN/Reporter/Smoker.pod#___top Smoke testing will download and run programs that other people have uploaded to CPAN. These programs could do *anything* to your system, including deleting everything on it. Do not run CPAN::Reporter::Smoker unless you are prepared to take these risks.
Re: rfc: safer smoking
On Sun, Jan 18, 2009 at 7:59 PM, David Westbrook dwestbr...@gmail.comwrote: Whereas you wouldn't want to run a smoker on a regular machine (e.g. a $work box, or your day-to-day windows desktop, or laptop, etc), this is reasonably safe since it's only testing distros that you already installed, which implies that they (and their dependencies) are trusted. Also lets you easily submit reports for all the modules you're actual using, w/o a separate smoke environment. I like it. Among other things, it could be used with the File transport to save reports to a directory for examination, which could be useful to check things before upgrading. Is this useful for others? If so, as a snippet (e.g. posted on wiki)? as a subclass (name??) of CPAN::Reporter::Smoker? as an additional exported sub (name??) in CPAN::Reporter::Smoker? At the very least, I'd put it on the CPAN Testers wiki. I have no problem if you want to publish a CPAN::Reporter::Smoker::* module, but I don't think I want to include it in C::R::Smoker itself, mostly because it's a slipperly slope of extra maintenance work for me if I start accepting different ways of generating the list. -- David
Re: rfc: safer smoking
thanks for the comments! Any suggestions for a good CPAN::Reporter::Smoker::* subclass name? --david On Sun, Jan 18, 2009 at 8:31 PM, David Golden xda...@gmail.com wrote: On Sun, Jan 18, 2009 at 7:59 PM, David Westbrook dwestbr...@gmail.com wrote: Whereas you wouldn't want to run a smoker on a regular machine (e.g. a $work box, or your day-to-day windows desktop, or laptop, etc), this is reasonably safe since it's only testing distros that you already installed, which implies that they (and their dependencies) are trusted. Also lets you easily submit reports for all the modules you're actual using, w/o a separate smoke environment. I like it. Among other things, it could be used with the File transport to save reports to a directory for examination, which could be useful to check things before upgrading. Is this useful for others? If so, as a snippet (e.g. posted on wiki)? as a subclass (name??) of CPAN::Reporter::Smoker? as an additional exported sub (name??) in CPAN::Reporter::Smoker? At the very least, I'd put it on the CPAN Testers wiki. I have no problem if you want to publish a CPAN::Reporter::Smoker::* module, but I don't think I want to include it in C::R::Smoker itself, mostly because it's a slipperly slope of extra maintenance work for me if I start accepting different ways of generating the list. -- David
