Re: how exactly to do unit testing in scheme...

[Christopher Lam]

Hi Robert and devel,

Thank you for pointers, this is useful. The examples below illustrate 
how to test multiple items together. But I think the complexity of the 
52(and counting) options (37 binary, the rest are multiples/multichoice) 
means, I think it'll be better to handwrite tests. I'll need to reuse 
existing test frameworks to attempt maximise coverage.


C

On 31/01/18 09:01, Robert Merkel wrote:

Sorry for the tardy response - I went away for the long weekend and was
busy yesterday!

As Phil says, you can test both leaf functions and controller code with
unit tests. With controller code, you may need to write mocks for some or
all of the called code.  I don't know whether there are prewritten mocking
libraries for guile, but given the extreme flexibility of scheme it
shouldn't be difficult to write code to replace one function call with
another where necessary.

As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
there is no particular reason from a testing perspective to throw in an
intermediate function.

As far as which options to test, if testing all the combinations
exhaustively results in too many tests (and it sounds like it does) try
pairwise combinatoric testing, which works as follows:

Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
can either be "on" or "off" it has two values).

For any pair of options o_x and o_y, for all the possible combinations of
option values there should be at least one test that has that combination.

To give a very simple case, for three options {a, b, c}  each of which can
take two values {off, on}, you could have the following tests:

  Option   a b c
Test
1 off  off   off
2 off   on  off
3 on   off   on
4 onon  off
5 off   offon

For the pair (a,b) tests 1 through 4 give all the possible combinations,
for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
combinations.

5 is the minimum number of tests that can meet the pairwise combinatoric
criterion, as compared to 8 if you tried all the possible combinations.
However, it becomes an even bigger win if you're trying lots of options:
you only need 9 tests for 8 options with two values, whereas you would need
256 tests to try every possible option combination!

To generate optimal pairwise test sets, you can use "jenny" (this is a
really useful but unmaintained tool, I really should take over the
maintenance):

http://burtleburtle.net/bob/math/jenny.html

Hope this helps, and feel free to ask more questions.  I will try to
respond more promptly!


On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam 
wrote:


Dear Devel



To rgmerk: Welcome back, and it was a nice to meet irl!



While simplifying transaction.scm and thinking of unit testing, I now have
a conundrum worthy of an expert view.



The reports require 2 main functions – the options generator and the
renderer; the options generator generates a options.scm controller object,
and the renderer takes options and outputs html.



I understand unit testing to handle testing of ‘leaf’ functions e.g.
(split->date), rather than the controller code (e.g. renderer takes options
and outputs html) – but to me this is rather silly because split->date only
tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
actual functionality.



With regards to unit testing I can see several issues



1. The refactored report has inlined most single-use functions into
lambda expressions – I figured that directly stating (xaccTransGetDate
(xaccSplitGetParent split)) is much more descriptive to a programmer than
to create a testable leaf function (split->date split). I can see the
benefits of both – leave as lambda expressions which will can be
understandable by anyone who is familiar with the API, or break them out
into 100s of single use functions which can be tested, but introduces a
whole layer of cognitive load to anyone hacking code – (what does
split->date actually do? Where is its definition). Also, breaking the
lambda functions into testable functions means the implementation is frozen
and the next hacker will have lesser scope to rework/optimise the report.



1. The refactored report is now flexible enough to accommodate derived
reports with a different multicolumn data function – eg
income-gst-statement.scm has been reworked into a transaction.scm
derivative which passes its own calculated-cells to report on GST sales and
purchases. This is not yet committed.



1. I think the most useful testing approach for a complex
transaction.scm will be to test functions of various combinations of
options values, and test the resulting html for 

Re: how exactly to do unit testing in scheme...

[Robert Merkel]

Sorry for the tardy response - I went away for the long weekend and was
busy yesterday!

As Phil says, you can test both leaf functions and controller code with
unit tests. With controller code, you may need to write mocks for some or
all of the called code.  I don't know whether there are prewritten mocking
libraries for guile, but given the extreme flexibility of scheme it
shouldn't be difficult to write code to replace one function call with
another where necessary.

As long as xaccTransGetDate and xaccSplitGetParent are adequately tested,
there is no particular reason from a testing perspective to throw in an
intermediate function.

As far as which options to test, if testing all the combinations
exhaustively results in too many tests (and it sounds like it does) try
pairwise combinatoric testing, which works as follows:

Say you've got options {o_1, o_2, o_3, ... o_n}, each of which can take a
limited set of values {o_i^1, o_i_2, o_i^k} (for instance, if the option
can either be "on" or "off" it has two values).

For any pair of options o_x and o_y, for all the possible combinations of
option values there should be at least one test that has that combination.

To give a very simple case, for three options {a, b, c}  each of which can
take two values {off, on}, you could have the following tests:

 Option   a b c
Test
1 off  off   off
2 off   on  off
3 on   off   on
4 onon  off
5 off   offon

For the pair (a,b) tests 1 through 4 give all the possible combinations,
for the pair (a, c) tests {1, 3, 4, 5} give all the possible combinations,
and for the pair (b, c) tests {1, 2, 3, 4} give all the possible
combinations.

5 is the minimum number of tests that can meet the pairwise combinatoric
criterion, as compared to 8 if you tried all the possible combinations.
However, it becomes an even bigger win if you're trying lots of options:
you only need 9 tests for 8 options with two values, whereas you would need
256 tests to try every possible option combination!

To generate optimal pairwise test sets, you can use "jenny" (this is a
really useful but unmaintained tool, I really should take over the
maintenance):

http://burtleburtle.net/bob/math/jenny.html

Hope this helps, and feel free to ask more questions.  I will try to
respond more promptly!


On Thu, Jan 25, 2018 at 3:03 AM, Christopher Lam 
wrote:

> Dear Devel
>
>
>
> To rgmerk: Welcome back, and it was a nice to meet irl!
>
>
>
> While simplifying transaction.scm and thinking of unit testing, I now have
> a conundrum worthy of an expert view.
>
>
>
> The reports require 2 main functions – the options generator and the
> renderer; the options generator generates a options.scm controller object,
> and the renderer takes options and outputs html.
>
>
>
> I understand unit testing to handle testing of ‘leaf’ functions e.g.
> (split->date), rather than the controller code (e.g. renderer takes options
> and outputs html) – but to me this is rather silly because split->date only
> tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
> actual functionality.
>
>
>
> With regards to unit testing I can see several issues
>
>
>
>1. The refactored report has inlined most single-use functions into
>lambda expressions – I figured that directly stating (xaccTransGetDate
>(xaccSplitGetParent split)) is much more descriptive to a programmer than
>to create a testable leaf function (split->date split). I can see the
>benefits of both – leave as lambda expressions which will can be
>understandable by anyone who is familiar with the API, or break them out
>into 100s of single use functions which can be tested, but introduces a
>whole layer of cognitive load to anyone hacking code – (what does
>split->date actually do? Where is its definition). Also, breaking the
>lambda functions into testable functions means the implementation is frozen
>and the next hacker will have lesser scope to rework/optimise the report.
>
>
>
>1. The refactored report is now flexible enough to accommodate derived
>reports with a different multicolumn data function – eg
>income-gst-statement.scm has been reworked into a transaction.scm
>derivative which passes its own calculated-cells to report on GST sales and
>purchases. This is not yet committed.
>
>
>
>1. I think the most useful testing approach for a complex
>transaction.scm will be to test functions of various combinations of
>options values, and test the resulting html for satisfactory output. There
>are now dozens of bools and multichoices that can be triggered, each
>effecting html in various ways. How best to test?
>
>
>
>1. My view would be the unit test would check that:
>   1. the TR actually exists
>   2. it can display empty-report
>   3. it can understand passing of 

Re: how exactly to do unit testing in scheme...

[Christopher Lam]

Thank you Phil, with help from IRC I'll be separating the renderer into
multiple testable steps.

Options to accountlist, to splits, to filtered splits, and finally to
table.

Each one uses different parts of options.

On 26 Jan 2018 1:55 AM, "Phil Longstaff"  wrote:

> Usually, unit testing controller code is done by writing mocks for the
> code that is called. In this case, this would be the options.scm controller
> and the renderer. The mock code would test that the expected arguments are
> passed, and would return a canned response. This both checks the logic of
> the controller but also allows error and other paths to be tested.
>
> On Wed, Jan 24, 2018 at 11:03 AM, Christopher Lam <
> christopher@gmail.com> wrote:
>
>> Dear Devel
>>
>> To rgmerk: Welcome back, and it was a nice to meet irl!
>>
>> While simplifying transaction.scm and thinking of unit testing, I now
>> have a conundrum worthy of an expert view.
>>
>> The reports require 2 main functions – the options generator and the
>> renderer; the options generator generates a options.scm controller object,
>> and the renderer takes options and outputs html.
>>
>> I understand unit testing to handle testing of ‘leaf’ functions e.g.
>> (split->date), rather than the controller code (e.g. renderer takes options
>> and outputs html) – but to me this is rather silly because split->date only
>> tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
>> actual functionality.
>>
>> With regards to unit testing I can see several issues
>>
>> 1) The refactored report has inlined most single-use functions into
>> lambda expressions – I figured that directly stating (xaccTransGetDate
>> (xaccSplitGetParent split)) is much more descriptive to a programmer than
>> to create a testable leaf function (split->date split). I can see the
>> benefits of both – leave as lambda expressions which will can be
>> understandable by anyone who is familiar with the API, or break them out
>> into 100s of single use functions which can be tested, but introduces a
>> whole layer of cognitive load to anyone hacking code – (what does
>> split->date actually do? Where is its definition). Also, breaking the
>> lambda functions into testable functions means the implementation is frozen
>> and the next hacker will have lesser scope to rework/optimise the report.
>>
>> 2) The refactored report is now flexible enough to accommodate derived
>> reports with a different multicolumn data function – eg
>> income-gst-statement.scm has been reworked into a transaction.scm
>> derivative which passes its own calculated-cells to report on GST sales and
>> purchases. This is not yet committed.
>>
>> 3) I think the most useful testing approach for a complex transaction.scm
>> will be to test functions of various combinations of options values, and
>> test the resulting html for satisfactory output. There are now dozens of
>> bools and multichoices that can be triggered, each effecting html in
>> various ways. How best to test?
>>
>> 4) My view would be the unit test would check that:
>> a. the TR actually exists
>> b. it can display empty-report
>> c. it can understand passing of custom-calculated-cells
>> d. each of the options can be toggled, and the resulting html
>> displays/hides cells/detail as expected
>> e. and sorting options generate sorted rows
>>
>> Comments welcome, I had no formal training ☹
>> ___
>> gnucash-devel mailing list
>> gnucash-devel@gnucash.org
>> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>>
>
>
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

Re: how exactly to do unit testing in scheme...

[Phil Longstaff]

Usually, unit testing controller code is done by writing mocks for the code
that is called. In this case, this would be the options.scm controller and
the renderer. The mock code would test that the expected arguments are
passed, and would return a canned response. This both checks the logic of
the controller but also allows error and other paths to be tested.

On Wed, Jan 24, 2018 at 11:03 AM, Christopher Lam  wrote:

> Dear Devel
>
> To rgmerk: Welcome back, and it was a nice to meet irl!
>
> While simplifying transaction.scm and thinking of unit testing, I now have
> a conundrum worthy of an expert view.
>
> The reports require 2 main functions – the options generator and the
> renderer; the options generator generates a options.scm controller object,
> and the renderer takes options and outputs html.
>
> I understand unit testing to handle testing of ‘leaf’ functions e.g.
> (split->date), rather than the controller code (e.g. renderer takes options
> and outputs html) – but to me this is rather silly because split->date only
> tests xaccTransGetDate and xaccSplitGetParent, whereas the controller tests
> actual functionality.
>
> With regards to unit testing I can see several issues
>
> 1) The refactored report has inlined most single-use functions into lambda
> expressions – I figured that directly stating (xaccTransGetDate
> (xaccSplitGetParent split)) is much more descriptive to a programmer than
> to create a testable leaf function (split->date split). I can see the
> benefits of both – leave as lambda expressions which will can be
> understandable by anyone who is familiar with the API, or break them out
> into 100s of single use functions which can be tested, but introduces a
> whole layer of cognitive load to anyone hacking code – (what does
> split->date actually do? Where is its definition). Also, breaking the
> lambda functions into testable functions means the implementation is frozen
> and the next hacker will have lesser scope to rework/optimise the report.
>
> 2) The refactored report is now flexible enough to accommodate derived
> reports with a different multicolumn data function – eg
> income-gst-statement.scm has been reworked into a transaction.scm
> derivative which passes its own calculated-cells to report on GST sales and
> purchases. This is not yet committed.
>
> 3) I think the most useful testing approach for a complex transaction.scm
> will be to test functions of various combinations of options values, and
> test the resulting html for satisfactory output. There are now dozens of
> bools and multichoices that can be triggered, each effecting html in
> various ways. How best to test?
>
> 4) My view would be the unit test would check that:
> a. the TR actually exists
> b. it can display empty-report
> c. it can understand passing of custom-calculated-cells
> d. each of the options can be toggled, and the resulting html
> displays/hides cells/detail as expected
> e. and sorting options generate sorted rows
>
> Comments welcome, I had no formal training ☹
> ___
> gnucash-devel mailing list
> gnucash-devel@gnucash.org
> https://lists.gnucash.org/mailman/listinfo/gnucash-devel
>
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel

how exactly to do unit testing in scheme...

[Christopher Lam]

Dear Devel 

To rgmerk: Welcome back, and it was a nice to meet irl!

While simplifying transaction.scm and thinking of unit testing, I now have a 
conundrum worthy of an expert view.

The reports require 2 main functions – the options generator and the renderer; 
the options generator generates a options.scm controller object, and the 
renderer takes options and outputs html.

I understand unit testing to handle testing of ‘leaf’ functions e.g. 
(split->date), rather than the controller code (e.g. renderer takes options and 
outputs html) – but to me this is rather silly because split->date only tests 
xaccTransGetDate and xaccSplitGetParent, whereas the controller tests actual 
functionality.

With regards to unit testing I can see several issues

1) The refactored report has inlined most single-use functions into lambda 
expressions – I figured that directly stating (xaccTransGetDate 
(xaccSplitGetParent split)) is much more descriptive to a programmer than to 
create a testable leaf function (split->date split). I can see the benefits of 
both – leave as lambda expressions which will can be understandable by anyone 
who is familiar with the API, or break them out into 100s of single use 
functions which can be tested, but introduces a whole layer of cognitive load 
to anyone hacking code – (what does split->date actually do? Where is its 
definition). Also, breaking the lambda functions into testable functions means 
the implementation is frozen and the next hacker will have lesser scope to 
rework/optimise the report.

2) The refactored report is now flexible enough to accommodate derived reports 
with a different multicolumn data function – eg income-gst-statement.scm has 
been reworked into a transaction.scm derivative which passes its own 
calculated-cells to report on GST sales and purchases. This is not yet 
committed.

3) I think the most useful testing approach for a complex transaction.scm will 
be to test functions of various combinations of options values, and test the 
resulting html for satisfactory output. There are now dozens of bools and 
multichoices that can be triggered, each effecting html in various ways. How 
best to test?

4) My view would be the unit test would check that:
a. the TR actually exists
b. it can display empty-report
c. it can understand passing of custom-calculated-cells
d. each of the options can be toggled, and the resulting html displays/hides 
cells/detail as expected
e. and sorting options generate sorted rows

Comments welcome, I had no formal training ☹
___
gnucash-devel mailing list
gnucash-devel@gnucash.org
https://lists.gnucash.org/mailman/listinfo/gnucash-devel