Re: [asdf-devel] ASDF test-op question

2009-11-09 Thread Tobias C. Rittweiler
This feature request is now logged as https://bugs.launchpad.net/asdf/+bug/479478 -T. ___ asdf-devel mailing list asdf-devel@common-lisp.net http://common-lisp.net/cgi-bin/mailman/listinfo/asdf-devel

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Juan Jose Garcia-Ripoll
Hi Robert, I hope you do not mind I CC this email to the list, because I think you made pretty good points and I think I also have some valid answers to them. 2009/10/19 Robert Goldman : > Yes, that's right.  You /do/ need to run through all of ASDF's code.  I > have been going through that code.

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Robert Goldman
Tobias C. Rittweiler wrote: > Robert Goldman writes: > >> Juan Jose Garcia-Ripoll wrote: >> >>> You are just imposing too much complexity. If I want to test package >>> Cl-UNICODE, I do nont want to test FLEXI-STREAMS or U-SOCKETS. Tests >>> should be atomic and not generate a tree of actions li

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Tobias C. Rittweiler
"Tobias C. Rittweiler" writes: > What if we start with the simple case of only carring one bit of > meta-information (successp), merging is nothing more than ORing together > the results. Bah, s/OR/AND/ -T. ___ asdf-devel mailing list asdf-devel@co

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Tobias C. Rittweiler
Robert Goldman writes: > Juan Jose Garcia-Ripoll wrote: > > > You are just imposing too much complexity. If I want to test package > > Cl-UNICODE, I do nont want to test FLEXI-STREAMS or U-SOCKETS. Tests > > should be atomic and not generate a tree of actions like ASDF does not > > for everythin

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Robert Goldman
Juan Jose Garcia-Ripoll wrote: > On Mon, Oct 19, 2009 at 12:00 AM, Robert Goldman wrote: >>> In particular because it's my impression that the problems have been >>> over-stated. > > I think so. And this is indicated by the following paragraphs [..snip..] > >> 2. One needs to come up with a me

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Juan Jose Garcia-Ripoll
On Mon, Oct 19, 2009 at 11:37 AM, Tobias C. Rittweiler wrote: > Juan Jose Garcia-Ripoll writes: >> Blocking this development just because there are 5 test suites and you >> do not know how to combine them with ASDF is really absurd. ASDF's >> specifications can not depend on what your company or o

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Tobias C. Rittweiler
Juan Jose Garcia-Ripoll writes: > On Mon, Oct 19, 2009 at 12:00 AM, Robert Goldman wrote: [for the record; the topmost paragraph came from me:] > > > In particular because it's my impression that the problems have been > > > over-stated. > > I think so. And this is indicated by the following par

Re: [asdf-devel] ASDF test-op question

2009-10-19 Thread Juan Jose Garcia-Ripoll
On Mon, Oct 19, 2009 at 12:00 AM, Robert Goldman wrote: >> In particular because it's my impression that the problems have been >> over-stated. I think so. And this is indicated by the following paragraphs > 1.  Not all regression test frameworks are functional, returning values. >  Some write r

Re: [asdf-devel] ASDF test-op question

2009-10-18 Thread Robert Goldman
Tobias C. Rittweiler wrote: > Robert Goldman writes: > >> Faré wrote: >> >>> Maybe ASDF is the wrong place to try to standardize testing infrastructure? >> This is the conclusion I have reached, as well. I was hoping that some >> very weak standard could be arrived at that would make the test-op

Re: [asdf-devel] ASDF test-op question

2009-10-18 Thread Tobias C. Rittweiler
Robert Goldman writes: > Faré wrote: > > > Maybe ASDF is the wrong place to try to standardize testing infrastructure? > > This is the conclusion I have reached, as well. I was hoping that some > very weak standard could be arrived at that would make the test-op more > generally useful to people

Re: [asdf-devel] ASDF test-op question

2009-10-09 Thread Robert Goldman
Faré wrote: > Maybe ASDF is the wrong place to try to standardize testing infrastructure? This is the conclusion I have reached, as well. I was hoping that some very weak standard could be arrived at that would make the test-op more generally useful to people installing systems, so that they coul

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Faré
Maybe ASDF is the wrong place to try to standardize testing infrastructure? I mean, maybe instead the authors of various test infrastructures should have a common list where they discuss interoperability, reporting, and a declarative way of specifying dependencies between test suites, between file

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Robert Goldman
Tobias C. Rittweiler wrote: > Robert Goldman writes: > >> Tobias C. Rittweiler wrote: >>> Robert Goldman writes: >>> 2. Returning a single operation isn't enough, is it? For example, if I have system X, with sub-systems A, B, and C, I may be testing A, B, and C, so my traversal wo

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Tobias C. Rittweiler
Robert Goldman writes: > Tobias C. Rittweiler wrote: >> Robert Goldman writes: >> >>> 2. Returning a single operation isn't enough, is it? For example, if I >>> have system X, with sub-systems A, B, and C, I may be testing A, B, and >>> C, so my traversal would have to gather up the three subsi

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Robert Goldman
Tobias C. Rittweiler wrote: > Robert Goldman writes: > >> 2. Returning a single operation isn't enough, is it? For example, if I >> have system X, with sub-systems A, B, and C, I may be testing A, B, and >> C, so my traversal would have to gather up the three subsidiary test-op >> entities and e

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Tobias C. Rittweiler
Robert Goldman writes: > 2. Returning a single operation isn't enough, is it? For example, if I > have system X, with sub-systems A, B, and C, I may be testing A, B, and > C, so my traversal would have to gather up the three subsidiary test-op > entities and either package them into the parent t

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Robert P. Goldman
The problem is that the asdf framework does not lend itself to returning things from ops (particularly if you need to roll up subsidiary results from sub-operations). ___ Robert P. Goldman Principal Scientist, SIFT, LLC www.sift.info . Original Message ... On Thu, 8 Oct 2009 12:18:50 +

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Robert P. Goldman
No, nobody's supposed to parse it. It's just so if you want you can look at *only* the output of the tester, not the output of the tester mooshed together with the output of the compiler. Our tester does a bunch of compilation and esp on sbcl, the test output proper gets mixed together with lot

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Nikodemus Siivola
2009/10/8 Tobias C. Rittweiler : > To me, the most interesting advantage that I see in ASDF providing a > test operation, is that it should allow for automatic testing of > arbitrary software packages. > > I do not see how providing a stream argument is relevant to that. Or do > you propose that p

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Juan Jose Garcia-Ripoll
On Thu, Oct 8, 2009 at 12:16 PM, Tobias C. Rittweiler wrote: > To me, the most interesting advantage that I see in ASDF providing a > test operation, is that it should allow for automatic testing of > arbitrary software packages. Indeed! See http://ecls.sourceforge.net/logs_lib.html > I do not s

Re: [asdf-devel] ASDF test-op question

2009-10-08 Thread Tobias C. Rittweiler
Robert Goldman writes: > An alternative solution would be to provide a :stream or :filename init > argument for the test-op operation class and bind a dynamic variable > around every perform, making the stream or filename available for > writing To me, the most interesting advantage that I se

Re: [asdf-devel] ASDF test-op question

2009-10-06 Thread Attila Lendvai
>> when test-op'ed, we print the result object to *standard-output*. when >> used interactively, the test defun that was used to start the testing >> simply returns the result value which we inspect in slime when needed. >> > > Do you see the same problems I see?  I find that my test's output can >

Re: [asdf-devel] ASDF test-op question

2009-10-06 Thread Robert Goldman
Attila Lendvai wrote: >> That is why I have been suggesting that we provide a test operation that >> binds a stream --- because most of the test frameworks I have worked >> with provide a test report, rather than returning results. > > fyi, stefil returns a CLOS object containing the test results

Re: [asdf-devel] ASDF test-op question

2009-10-06 Thread Attila Lendvai
> That is why I have been suggesting that we provide a test operation that > binds a stream --- because most of the test frameworks I have worked > with provide a test report, rather than returning results. fyi, stefil returns a CLOS object containing the test results (and provides slime inspector

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Robert Goldman
Daniel Herring wrote: > On Mon, 5 Oct 2009, Robert Goldman wrote: >> Daniel Herring wrote: >>> On Mon, 5 Oct 2009, Robert Goldman wrote: Gary King wrote: > Hi Robert, > >> I don't believe that this is a general solution, for two reasons: > I agree that it isn't a general soluti

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Daniel Herring
On Mon, 5 Oct 2009, Robert Goldman wrote: > Daniel Herring wrote: >> On Mon, 5 Oct 2009, Robert Goldman wrote: >>> Gary King wrote: Hi Robert, > I don't believe that this is a general solution, for two reasons: I agree that it isn't a general solution especially since there is no

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Robert Goldman
Daniel Herring wrote: > On Mon, 5 Oct 2009, Robert Goldman wrote: >> Gary King wrote: >>> Hi Robert, >>> I don't believe that this is a general solution, for two reasons: >>> I agree that it isn't a general solution especially since there is no >>> interface/API for clients to do anything with

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Daniel Herring
On Mon, 5 Oct 2009, Robert Goldman wrote: > Gary King wrote: >> Hi Robert, >> >>> I don't believe that this is a general solution, for two reasons: >> >> I agree that it isn't a general solution especially since there is no >> interface/API for clients to do anything with an ASDF operation! It >> m

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Robert Goldman
Gary King wrote: > Hi Robert, > >> I don't believe that this is a general solution, for two reasons: > > I agree that it isn't a general solution especially since there is no > interface/API for clients to do anything with an ASDF operation! It > might, however, be a small step in the right direc

Re: [asdf-devel] ASDF test-op question

2009-10-05 Thread Gary King
Hi Robert, > I don't believe that this is a general solution, for two reasons: I agree that it isn't a general solution especially since there is no interface/API for clients to do anything with an ASDF operation! It might, however, be a small step in the right direction. -- Gary Warren Kin