If you don't intend to keep the old business logic in the long run,
perhaps a version control system such as Git can help you. If you use it
in single-user mode, you can think of it as a backup system where you
manually create each snapshot and give it a name, but it actually can do
much more. For your use case, you can open a new *branch* where you
implement your changes, and implement your testing logic simultaneously
in both branches (using *merge* operations). The system handles
switching between branches, so you can really perform invasive changes,
and revert if you find that a particular change breaks something.
RStudio has Git support, but you probably need to use the shell to
create a branch. On Windows or OS X the GitHub client helps you to get
started.
Cheers
Kirill
On 09/10/2014 11:14 AM, Stephanie Locke wrote:
I have unit tests using testthat but these are typically of these types:
1) Check for correct calculation for a single set of valid inputs
2) Check for correct calculation for a larger set of valid inputs
3) Check for errors when providing incorrect inputs
4) Check for known frailties / past issues
This is more for where changes are needed to functions that apply various bits of
business logic that can change over time, so there is no "one answer". A unit
test (at least as I understand it) can be worked through to make sure that given inputs,
the output is computationally correct. What I'd like to do is overall the impact of a
potential change by testing version 1 of a function in a package for a sample, then test
version 2 of a function in a package for a sample and compare the results.
My difficulties encountered so far is I'm reluctantly to manually do this
change invasively by overwriting the relevant files in the R directory, and
then say using devtools to load it and test it with testthat as I risk
producing incorrect states of my package and potentially releasing the wrong
thing. My preference would be a non-invasive method. Currently, where I'm
trying to do this non-invasively I source a new version of the function stored
in a separate directory, but some of the functions dependent on it continue to
reference to the package version of the functions, this means that when I'm
doing test #2 I have to load lots more functions and hope I've caught them all
(or do some sort of dependency hunting programmatically).
I may be missing something about testthat, but what I'm doing now seems to be
nowhere near optimal and I'd love to have a better solution.
Cheers
Stephanie Locke
BI & Credit Risk Analyst
-----Original Message-----
From: ONKELINX, Thierry [mailto:thierry.onkel...@inbo.be]
Sent: 10 September 2014 09:30
To: Stephanie Locke; r-devel@r-project.org
Subject: RE: How to test impact of candidate changes to package?
Dear Stephanie,
Have a look at the testthat package and the related article in the R Journal.
Best regards,
ir. Thierry Onkelinx
Instituut voor natuur- en bosonderzoek / Research Institute for Nature and Forest team
Biometrie & Kwaliteitszorg / team Biometrics & Quality Assurance Kliniekstraat
25
1070 Anderlecht
Belgium
+ 32 2 525 02 51
+ 32 54 43 61 85
thierry.onkel...@inbo.be
www.inbo.be
To call in the statistician after the experiment is done may be no more than
asking him to perform a post-mortem examination: he may be able to say what the
experiment died of.
~ Sir Ronald Aylmer Fisher
The plural of anecdote is not data.
~ Roger Brinner
The combination of some data and an aching desire for an answer does not ensure
that a reasonable answer can be extracted from a given body of data.
~ John Tukey
-----Oorspronkelijk bericht-----
Van: r-devel-boun...@r-project.org [mailto:r-devel-boun...@r-project.org]
Namens Stephanie Locke
Verzonden: woensdag 10 september 2014 9:55
Aan: r-devel@r-project.org
Onderwerp: [Rd] How to test impact of candidate changes to package?
I use a package to contain simple functions that can be handled by unit tests
for correctness and more complex functions that combine the simple functions
with business logic. Where there are proposals to change either the simple
functions or the business logic, a sample needs to be run before the change and
then after it to understand the impact of the change.
I do this currently by
1. Using Rmarkdown documents
2. Loading the package as-is
3. Getting my sample
4. Running my sample through the package as-is and outputting table of results
5. sourceing new copies of functions 6. Running my sample again and outputting
table of results 7. Reloading package and sourceing different copies of
functions as required
I really don't think this is a good way to do this as it risks missing
downstream dependencies of the functions I'm trying to load into the global
namespace to test.
Has anyone else had to do this sort of testing before on their packages? How
did you do it? Am I missing an obvious package / framework that can do this?
Cheers,
Steph
--
Stephanie Locke
BI & Credit Risk Analyst
The contents of this e-mail and of any attachments transmitted with it are
confidential and intended solely for the use of the individual(s) to whom they
are addressed. If you are not an intended recipient or the person responsible
for delivering this e-mail to the intended recipient(s), you have received this
e-mail in error, and access to it by you is not authorised. You may not use,
copy, distribute, disclose or rely on it or any attachment, or any part of it
or any attachment in any way. If you have received this e-mail in error please
notify Optimum Credit Ltd by email on webmas...@optimumcredit.co.uk or phone on
03330 143125 and then delete it.
All reasonable precautions have been taken to ensure that no viruses are
present in E-mail. As we cannot accept responsibility for loss or damage
arising from the use of E-mail or any attachment we recommend you subject these
to your usual virus checking procedures prior to use. Any opinions expressed in
this e-mail are those of the person sending it and not necessarily those of
Optimum Credit Ltd.
Optimum Credit Ltd, Haywood House South, Dumfries Place, Cardiff, CF10 3GA
Authorised and regulated by the Financial Conduct Authority
Optimum Credit Ltd. Registered office: Haywood House South, Dumfries Place,
Cardiff, CF10 3GA. Registered in England and Wales No. 08698121
Calls may be monitored or recorded for training, compliance and evidential
purposes.
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
* * * * * * * * * * * * * D I S C L A I M E R * * * * * * * * * * * * * Dit
bericht en eventuele bijlagen geven enkel de visie van de schrijver weer en
binden het INBO onder geen enkel beding, zolang dit bericht niet bevestigd is
door een geldig ondertekend document.
The views expressed in this message and any annex are purely those of the
writer and may not be regarded as stating an official position of INBO, as long
as the message is not confirmed by a duly signed document.
The contents of this e-mail and of any attachments transmitted with it are
confidential and intended solely for the use of the individual(s) to whom they
are addressed. If you are not an intended recipient or the person responsible
for delivering this e-mail to the intended recipient(s), you have received this
e-mail in error, and access to it by you is not authorised. You may not use,
copy, distribute, disclose or rely on it or any attachment, or any part of it
or any attachment in any way. If you have received this e-mail in error please
notify Optimum Credit Ltd by email on webmas...@optimumcredit.co.uk or phone on
03330 143125 and then delete it.
All reasonable precautions have been taken to ensure that no viruses are
present in E-mail. As we cannot accept responsibility for loss or damage
arising from the use of E-mail or any attachment we recommend you subject these
to your usual virus checking procedures prior to use. Any opinions expressed in
this e-mail are those of the person sending it and not necessarily those of
Optimum Credit Ltd.
Optimum Credit Ltd, Haywood House South, Dumfries Place, Cardiff, CF10 3GA
Authorised and regulated by the Financial Conduct Authority
Optimum Credit Ltd. Registered office: Haywood House South, Dumfries Place,
Cardiff, CF10 3GA. Registered in England and Wales No. 08698121
Calls may be monitored or recorded for training, compliance and evidential
purposes.
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
