Re: Dual life t/test.pl?
On 2011.11.17 4:08 AM, Nicholas Clark wrote: 1) It's not really my goal to make it distributable as a CPAN module, but as just something you copy. It continues to be my goal to reduce the amount of effort needed to support the core. Exposing more of the internals runs counter to this. 2) Test::More would lose the benefit of improvements from p5p. 3) p5p would lose the benefit of improvements from Test::More. The core is using Test::More for pretty much every core-maintained test outside of t/ One of the things I've been working on slowly is trying to turn pretty much to every, by removing each impediment in turn. p5p continues to benefit from Test::More, and certainly I'm grateful for your continued work on that. (I assume everyone else is) Sorry, I think I wasn't clear. It wasn't meant as a I'm taking Test::More and going home screed. I was only commenting on FC's suggestion to fork t/test.pl. That is, when Test::More patches its copy of t/test.pl p5p can benefit. And when p5p patches t/test.pl Test::More can benefit. That only works if I keep them in sync. The library upon which 372 core tests rely is undocumented and untested. Moving it into its own repo and tracker allows it to be tested and stable without having to go through the p5p memorial bike shed. That's the entire bloody point. It's not *meant* to be assumed to be stable. It's an internal implementation detail of testing the core. I think this is the key point on which we fundamentally disagree, and we can't go further without clearing this one up. If I'm hearing you correctly, your argument is that t/test.pl is internal to the core and does not need to be documented or tested. Docs are for publishing stable APIs. Tests are for checking regression against those stable APIs. My argument is that in order to write tests using t/test.pl, core developers need to know how to use it. That's what docs would be for. What does fresh_perl() do, what are it's arguments and limitations? What about run_multiple_progs()? display()? within()? like_yn()? That t/test.pl lacks documentation is a barrier to core developers writing tests. Just because it's internal to the core doesn't mean people magically know how to use it. Tests for t/test.pl have a similar argument. They make sure that the basis for our core testing system works and continues to work. They ensure that changes to how it works are deliberate and not accidental. Docs and tests are something t/test.pl needs, dual-life or not. If we don't agree on that, then this isn't going anywhere. -- 29. The Irish MPs are not after Me frosted lucky charms. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/list/
Re: Dual life t/test.pl?
On Wed, Nov 16, 2011 at 11:15:21PM -0800, Michael G Schwern wrote: On 2011.11.16 5:58 PM, Father Chrysostomos via RT wrote: I think these patches are safe, but given the recent thread about dual-lifing test.pl I'm not even sure whether I should apply it or not. (Nicholas Clark does seem to have some good points.) This patch is not affected by that discussion. [Agree. Will try to get back to that in a different message] I don't really want to get involved in the thread, but let me ask you one thing (and I'll let others [e.g., rjbs] respond to your response): Why not simply fork test.pl? That way you can strip out the miniperl-specific stuff and have an even more lightweight testing library. You could even pop it in a .pm file, add package foo and @EXPORT, and with little effort you have a shiny new Test::More::Lite module. 1) It's not really my goal to make it distributable as a CPAN module, but as just something you copy. It continues to be my goal to reduce the amount of effort needed to support the core. Exposing more of the internals runs counter to this. 2) Test::More would lose the benefit of improvements from p5p. 3) p5p would lose the benefit of improvements from Test::More. The core is using Test::More for pretty much every core-maintained test outside of t/ One of the things I've been working on slowly is trying to turn pretty much to every, by removing each impediment in turn. p5p continues to benefit from Test::More, and certainly I'm grateful for your continued work on that. (I assume everyone else is) 4) I already maintain enough test modules, thanks. [referred to below] On Tue, Nov 15, 2011 at 06:16:37PM -0800, Michael G Schwern wrote: First thing to clarify, I don't really envision this being turned into a CPAN module. That would be directly against its purpose. The primary purpose here is to separate and focus development and provide a point for module authors to copy it into their project. But that's not the point of t/test.pl It's not meant as a lightweight general purpose testing system. It's intended to test miniperl and low level Perl features, without assuming that everything is actually working. That it turns out to be a lightweight test system is, well, convenient. Convenient for other people. It probably will *not* be released to CPAN and it probably will *not* be API stable. The needs of the core outweigh the needs of the few people that use it. Totally agree about the needs of the core. But not on the interpretation. The continued experience of the core is that a: for any part that anyone can access, someone starts to rely on it. b: because anyone can read the core's source, they can see non-accessible parts that would be useful. But these parts were *never* written with any consideration to whether they make a sane *public* API. Historically what has typically happened is that someone has sent in a patch to expose it, that's cool (as everyone thinks about how they too could use it) and it's applied, as a nuclear reactor in bikeshed terms. Which ends up with ever more parts of the core exposed, replied upon, and implicitly supported. Right now, t/test.pl is not installed by the core. So it's an internal implementation detail of testing. Anyone else who uses it copied it, and by act of copying it to install it they've taken an action that acknowledges that they're violating a form of public/private encapsulation. On 2011.11.15 6:34 AM, Nicholas Clark wrote: Other than people writing test modules, who would find it useful? Test::More is about to become a lot more complicated internally, so anyone who does things which Test::More might be using. In addition, .t files which run a bazillion tests would find it useful both for speed and memory usage. I just recommended Buddy Burden use it for some huge Data::Random tests. Abigail could use it for her enormous Regexp::Common test suites. But that's not really what it's about. The library upon which 372 core tests rely is undocumented and untested. Moving it into its own repo and tracker allows it to be tested and stable without having to go through the p5p memorial bike shed. That's the entire bloody point. It's not *meant* to be assumed to be stable. It's an internal implementation detail of testing the core. It does what *we* need quite nicely. We're not looking for patches for it to make it into something else. The p5p memorial bike shed *is* a problem (both for bikesheds and nuclear reactors, and everything in between) but assuming that it's a problem *here* is based on the axiom that t/test.pl is something that *should* be externally visible. An axiom which I reject. It's a lot easier to get people to work on a single file Perl library than it is to get them to patch the Perl core. In its own project, I can hand a newbie a URL like this and say pick one of these and do it!
Re: Dual life t/test.pl?
This was suggested previously in 2008 http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-06/msg00912.html On Mon, Nov 14, 2011 at 04:59:57PM -0800, Michael G Schwern wrote: Having a parallel and featureful testing system is very useful. I use it to test Test::More (in the Test-Builder1.5 branch). Others might find it useful to do the same. Other than people writing test modules, who would find it useful? t/test.pl is very important to the Perl core tests, but it is largely undocumented and untested. Going through the p5p process to patch it makes working on it difficult and open to bike shedding. If it were dual-lived it could be worked on as its own thing with its own commits and tests. test.pl is a script, not a package. Our machinery for dealing with dual life things is really only set up to deal with modules. So doing this is not going to just drop in to what we have. By having a dedicated feed of issues and commits, careful attention could be paid to what code and features are going in. Close attention and discussion can happen about whether the code is simple enough and the features apt for the needs of the core. Surely the needs of the core *are* better determined by the core developers, not some third party group? Also, having upstream be anywhere other than the core actively gets in the way of t/test.pl actually being useful to core developers (or at least to me, as a core developer who has added things to it). Even having it being upstream blead gets in the way of it being useful. Right now, as all users of the core's t/test.pl are in the core, it's possible to a: add things to it without stopping to think is this a decent, supportable API that can last forever, because if the API turns out to be wrong, it's entirely possible to have a flag day commit that changes it and all its users b: remove things from it, because nothing is using them any more c: not worry about what versions of perl it supports, or working round problems in them (Although to be fair, if a particular piece of perl syntax had been buggy before, it's possibly something fragile that should be avoided in case it breaks again for a different reason) Nothing in the above stops anyone taking a copy of test.pl and using it. Doing this also doesn't add a layer of process into working on the core. If folks are cool with that, I can set up a Github repository for it. We'll worry about whether CPAN release is appropriate later. I'm distinctly uncool about doing it that way. I'm unconvinced about doing it with blead upstream. This feels like a lot of work for little benefit, with the work outweighing the benefit. We did start work on trying to extract runperl() as Test::Builder module. There's a branch from blead where I attempted to convert module tests to use it and found which features were necessary for them: http://perl5.git.perl.org/perl.git/shortlog/refs/heads/smoke-me/runperl Beware that IPC::Cmd can't capture output on Win32 with core-only modules, and IPC::Run can't reliably capture output on Win32, thanks to Microsoft's socketpair implementation following advice from Klortho: #11912 Evidently it's important to you to get the wrong answer as quickly as possible. (The default for closing a socket on Win32 is to drop all unsent buffered data on the floor. No, I'm not kidding. Note To assure that all data is sent and received on a connection, an application should call shutdown before calling closesocket (see Graceful shutdown, linger options, and socket closure for more information). Also note, an FD_CLOSE network event is not posted after closesocket is called. http://msdn.microsoft.com/en-us/library/ms737582%28v=vs.85%29.aspx I've also found http://msdn.microsoft.com/en-us/library/ms738547%28v=vs.85%29.aspx Whereas the BSD socket() implementation works in such a way that a socket FD can be substituted for a file FD, pipe FD or tty FD, and the program continues to work just as it did before, without needing any special case code. Doesn't even need recompiling.) Nicholas Clark
Re: Dual life t/test.pl?
On Nov 15, 2011, at 6:43 AM, Jerry D. Hedden wrote: Other than people writing test modules, who would find it useful? FWIW, threads and threads::shared use test.pl in their test suites. This seems to be historical and related to the fact that older versions of Test::More didn't work will with threads. However, changing to use Test::More may be undesirable because it means testing threads using a module that needs to support threads testing. The DBI uses it, too, for performance testing, I believe. http://svn.perl.org/viewvc/modules/dbi/trunk/test.pl?view=markup Best, David
Re: Dual life t/test.pl?
On Tue, Nov 15, 2011 at 09:34, Nicholas Clark n...@ccl4.org wrote: This was suggested previously in 2008 http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2008-06/msg00912.html On Mon, Nov 14, 2011 at 04:59:57PM -0800, Michael G Schwern wrote: Having a parallel and featureful testing system is very useful. I use it to test Test::More (in the Test-Builder1.5 branch). Others might find it useful to do the same. Other than people writing test modules, who would find it useful? FWIW, threads and threads::shared use test.pl in their test suites. This seems to be historical and related to the fact that older versions of Test::More didn't work will with threads. However, changing to use Test::More may be undesirable because it means testing threads using a module that needs to support threads testing.
