Reference data while testing - interesting

[Mike Dewhirst]

I have an interesting testing problem which requires someone smarter 
than me.


I want to prove correctness of multiple outcomes from multiple 
regulatory computations based on, among other things, international 
lists of assessed chemicals and their hazards.


Hazards include both human health and environmental - land, air and 
aquatic as well as the food chain from microbes on up.


Once a qualified chemistry expert has agreed that our (Django) software 
has correctly categorised a chemical according to local regulations and 
those lists, we need to lock in that behaviour which, as already 
indicated, is based on reference data.


To do so, we wrote a (Django) management command to auto-generate a set 
of tests for that chemical which proves all those outcomes are as 
expected whenever we run our tests. That protects against unintended 
side effects as we develop but not against changes in the reference data.


The interesting problem is that things change. Every day, new 
discoveries are made which indicate individual chemicals or chemical 
groups are actually more hazardous than previously thought.


Reference data fixtures are infeasible (dozens of tables, hundreds of 
thousands of records) because they are enormous and therefore 
unmanageable for frequent testing. Even if we did use fixtures, if not 
continuously refreshed, tests just keep passing.


We need tests to fail when change happens so that our software can 
notify users that they have to rethink what they might be doing with a 
particular chemical or group of chemicals in order to avoid further 
damaging the planet or its inhabitants.


The base reference data is updated when things change. These are 
necessarily manual processes based on subscribing to emails to receive 
notifications from local and international regulators about upcoming 
changes, calls for public input and eventually publishing of actual 
adjustments and deadline dates. That is a continuous process.


I put this scenario to ChatGPT and prompted it down the db_alias path[1] 
of keeping the reference data portion of the database available 
read-only during testing so that tests are directly exposed to the 
source of truth. It came up with a complicated/complex solution which, 
due to underconfidence, I have not tried.


I cannot believe this is the first time a reference data source has been 
needed during Django testing so I'm asking here if anyone has a solution.


Many thanks

Cheers

Mike


- - - - -
[1] Given the specific requirements of your testing scenario, where you 
need to access read-only reference data from a production database while 
emptying the regular testing database between tests, you can achieve 
this by using Django's |db_alias| feature. You can configure a separate 
database alias for read-only access to the reference data.


Here's how you can set it up:

1.

   *Configure a Separate Database for Reference Data*:

   In your Django settings (|settings.py|), define a new database
   configuration for the reference data. You can specify this in the
   |DATABASES| setting using a different alias:

#

|DATABASES = { 'default': { # Your regular database configuration for 
testing 'ENGINE': 'django.db.backends.postgresql', 'NAME': 'test_db', # 
Other settings... }, 'reference_db': { # Configuration for the reference 
data database (read-only) 'ENGINE': 'django.db.backends.postgresql', 
'NAME': 'reference_db', # Other settings... }, } |


Replace |'reference_db'| with an appropriate alias for your reference data.

#

*Create a Router for Reference Data*:

Next, you can create a database router to determine which database to 
use based on the model being accessed. In your Django project, create a 
Python module (e.g., |routers.py|) and define a database router class:


#

|class ReferenceDataRouter: def db_for_read(self, model, **hints): # Use 
the 'reference_db' alias for models that access reference data if 
model._meta.app_label == 'your_app_label': return 'reference_db' return 
None def db_for_write(self, model, **hints): # Prevent writes to the 
'reference_db' database if model._meta.app_label == 'your_app_label': 
return None return 'default' def allow_migrate(self, db, app_label, 
model_name=None, **hints): # Allow migrations only on the default 
database if db == 'default': return True return False |


Replace |'your_app_label'| with the actual app label containing your 
reference data models.


#

*Add the Router to Your Settings*:

In your Django settings (|settings.py|), add the reference data router 
to the |DATABASE_ROUTERS| setting:


#

|DATABASE_ROUTERS = ['your_project.routers.ReferenceDataRouter'] |

#

*Write Tests That Access Reference Data*:

In your test cases, you can access the reference data by using the 
|using| method to specify the database alias:


4.

   |from django.test import TestCase from your_app.models import
   YourReferenceModel class YourTestCase(TestCase): def
   test_access_reference_data(self): reference_objects =

Re: Reference data while testing

[Mike Dewhirst]

Thank you but unfortunately, none of the suggestions are appropriate. 
Here is more information.


We have a complex system where - in the opinion of a chemical expert - 
it is possible to state that a particular chemical risk profile has been 
correctly calculated by the software.


In addition to molecular properties, base and comparison information 
used bythe software to perform those risk calculations - at multiple 
end-points - is imported from many international regulatory authorities 
and stored in (Django/Postgres) reference tables. There are hundreds of 
thousands of records all of which are (somewhat) regularly updated when 
those authorities notify changes.


Once an expert confirms a risk profile for a particular chemical we use 
a Django management command to generate a new Django test in our test 
harness. The objective is to run that/those tests in our regular testing 
regime to prove the software - and reference data - is still OK.


If the reference data or the software changes, we want to discover any 
discrepancies from the expert opinion.


Automating creation of those tests (underpinning the expert opinion) is 
important because it is the only way I can see to stay sane. I am not a 
chemical expert.


I was hoping someone could suggest a way - perhaps using dbalias - to 
allow read-only use of the reference portion of the database. Another 
thought I had was to load a dump of the reference data prior to starting 
the tests but I believe Django would simply wipe that after the first test.


Factory boy is really good but cannot provide accurate reference 
information. I can see SQLAlchemy might be a solution but I'd prefer to 
stay within Django/Postgres.


Still looking ...

Thanks

On 4/10/2023 5:26 am, Alessandro Madruga Correia wrote:



On Mon, Oct 2, 2023 at 11:30 PM Mike Dewhirst  
wrote:



This is probably in the docs so please drop me a link if you can.

My project has fifteen or more tables of reference data which
determine how the software behaves and therefore is critical to
unit testing.

Fixtures are unworkable and I  need to respond to the actual
reference data.

How can i do real lookups (read-only) during tests?


You can use Factory Boy https://factoryboy.readthedocs.io/en/stable/


--
  ,= ,-_-. =. [] Alessandro Madruga Correia
 ((_/)o o(\_))  [http://counter.li.org]       Debian User# 342751
   `-'(. .)`-'    "O fanatismo é a única forma de força de vontade
       \_/        acessível aos fracos." (Friedrich Nietzsche)

--
You received this message because you are subscribed to the Google 
Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send 
an email to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAOofabfi%3DErk535OgDo5nPU%3DwM1pVp%3D93muppuGMt1c6M2vSZA%40mail.gmail.com 
.



--
Signed email is an absolute defence against phishing. This email has
been signed with my private key. If you import my public key you can
automatically decrypt my signature and be sure it came from me. Your
email software can handle signing.

--
You received this message because you are subscribed to the Google Groups "Django 
users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/d50a02f1-bcf9-7ac7-0488-95cc1d54eccd%40dewhirst.com.au.


OpenPGP_signature
Description: OpenPGP digital signature

Re: Reference data while testing

[Qayyam Hussain]

Sure I'll what how much you pay me?

On Tue, Oct 3, 2023, 7:31 AM Mike Dewhirst  wrote:

>
> This is probably in the docs so please drop me a link if you can.
>
> My project has fifteen or more tables of reference data which determine
> how the software behaves and therefore is critical to unit testing.
>
> Fixtures are unworkable and I  need to respond to the actual reference
> data.
>
> How can i do real lookups (read-only) during tests?
>
> Thanks
>
> Mike
>
>
>
>
> --
> (Unsigned mail from my phone)
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/651b7cc1.170a0220.54b8d.3ebcSMTPIN_ADDED_MISSING%40gmr-mx.google.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAOx6qv-kapk%2Bxm8%3DBzpF32M1JveZ01Xk4SaBts%2BxLMjEg5_k8g%40mail.gmail.com.

Re: Reference data while testing

[Alessandro Madruga Correia]

On Mon, Oct 2, 2023 at 11:30 PM Mike Dewhirst  wrote:

>
> This is probably in the docs so please drop me a link if you can.
>
> My project has fifteen or more tables of reference data which determine
> how the software behaves and therefore is critical to unit testing.
>
> Fixtures are unworkable and I  need to respond to the actual reference
> data.
>
> How can i do real lookups (read-only) during tests?
>

You can use Factory Boy https://factoryboy.readthedocs.io/en/stable/


-- 
  ,= ,-_-. =.   [] Alessandro Madruga Correia
 ((_/)o o(\_))  [http://counter.li.org]   Debian User# 342751
   `-'(. .)`-'"O fanatismo é a única forma de força de vontade
   \_/acessível aos fracos." (Friedrich Nietzsche)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAOofabfi%3DErk535OgDo5nPU%3DwM1pVp%3D93muppuGMt1c6M2vSZA%40mail.gmail.com.

Re: Reference data while testing

[herve bineli]

Hi Mike,

I will suggest you use a database framework like SQLAlchemy to access your
tables' data with read-only queries.
Then you use the data queried to set your test cases using a testing
library like Pytest.

Do let know if you have other concerns with this approach.

Best Regards,
BINELI Arsene


On Tue, Oct 3, 2023 at 3:30 AM Mike Dewhirst  wrote:

>
> This is probably in the docs so please drop me a link if you can.
>
> My project has fifteen or more tables of reference data which determine
> how the software behaves and therefore is critical to unit testing.
>
> Fixtures are unworkable and I  need to respond to the actual reference
> data.
>
> How can i do real lookups (read-only) during tests?
>
> Thanks
>
> Mike
>
>
>
>
> --
> (Unsigned mail from my phone)
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/651b7cc1.170a0220.54b8d.3ebcSMTPIN_ADDED_MISSING%40gmr-mx.google.com
> 
> .
>


-- 
*BINELI MANGA Hervé Arsène*
Ingénieur Informaticien - ENSPY
binelima...@gmail.com
(+237) 691388922 / 699946323

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAJm63Om2A-EwJdm7ck60rOOK%2By7kQCoSsaLb-mSmwzWAvM%2B92w%40mail.gmail.com.

Re: Reference data while testing

[Jun Say]

Thank you for your email.
https://hahaha0505.netlify.app/
If  you have any questions, please share with me.
Thank you.

On Tue, Oct 3, 2023 at 10:30 AM Mike Dewhirst  wrote:

>
> This is probably in the docs so please drop me a link if you can.
>
> My project has fifteen or more tables of reference data which determine
> how the software behaves and therefore is critical to unit testing.
>
> Fixtures are unworkable and I  need to respond to the actual reference
> data.
>
> How can i do real lookups (read-only) during tests?
>
> Thanks
>
> Mike
>
>
>
>
> --
> (Unsigned mail from my phone)
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-users/651b7cc1.170a0220.54b8d.3ebcSMTPIN_ADDED_MISSING%40gmr-mx.google.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/CAJyUeCSv4iEGViXc%2B2%2BGP4TyA7RaBzV1P%2B0A731HLJDPTB8--w%40mail.gmail.com.

Reference data while testing

[Mike Dewhirst]

This is probably in the docs so please drop me a link if you can.My project has 
fifteen or more tables of reference data which determine how the software 
behaves and therefore is critical to unit testing.Fixtures are unworkable and I 
 need to respond to the actual reference data.How can i do real lookups 
(read-only) during tests?ThanksMike--(Unsigned mail from my phone)

-- 
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-users/651b7cc1.170a0220.54b8d.3ebcSMTPIN_ADDED_MISSING%40gmr-mx.google.com.