On Oct 25, 2011, at 1:26 PM, F.J. Kraan wrote:
On 2011-10-25 19:01, Hans-Christoph Steiner wrote:
On Oct 25, 2011, at 11:44 AM, F.J. Kraan wrote:
On Mon, Oct 24, 2011 at 7:35 PM, Hans-Christoph Steiner<h...@at.or.at
> wrote:
Ah, course, makes sense. The third item there, the IIR
filters, it
should be not too hard to reproduce the exact same operation
with them
too. With the tests, each one is run in a new Pd instance, so
they're
always starting from scratch. Pd is then quit, and restarted
for the
next test.
But how do you want to create and check the test if the only
environment the test will produce proper results is the unattended
automatic run? If the test would fail, it would be hard to find
out why, as the manual run would always produce a different result.
I guess I don't quite follow what you mean here. I am thinking
that for a more elaborate test, the test patch would would run thru
the IIR filter a few times. Each iteration of the IIR filter test
should produce the same result.
Ok, I see. Here you are testing for repeatability. As an addition to
a test for regression.
One of the 'requirements' for the test-patch was something that
makes it easy to create tests, validate and run them, and find the
cause if the test fails.
Yes, making it easy is the primary concern right now so we can get
lots of tests written and running. But for very specific cases like
the IIR case, we would need to test for repeatability of multiple
iterations. Pd is designed to be 'deterministic', meaning that a
patch should produce the same result every time its run, so things
should be very repeatable.
This does remind me tho, the original load_every_help.py script
would load each help patch into the same instance of Pd, just
continually reusing it. That would trigger a couple hard-to-
reproduce bugs that basically only happened when loading every help
patch in a certain order. I was focused on getting a report on each
help patch, so I changed the script to make a new Pd instance per
test. But Pd should be able to load every help patch without
crashing, so perhaps we need a second test mode where the same Pd
is reused again and again.
The possibilities are endless here. We should try to stick to a
sensible subset of these and focus on that. Even then the test set
can grow larger that Pd itself.
Yes, definitely lets limit the scope to get tests running first. I
was just thinking there could be two modes for running the tests:
every test in a single Pd instance; and, each test in its own Pd
instance. I have one of those cases working, doing the other would
not be too hard, I think. But yes, lower priority.
.hc
----------------------------------------------------------------------------
"A cellphone to me is just an opportunity to be irritated wherever you
are." - Linus Torvalds
_______________________________________________
Pd-dev mailing list
Pd-dev@iem.at
http://lists.puredata.info/listinfo/pd-dev
