2014-11-12 0:25 GMT+01:00 Carl Meyer <c...@oddbird.net>:
> Hi David,
>
> On 11/11/2014 08:37 AM, dpalao.pyt...@gmail.com wrote:
>> Dear Carl,
>>
>> Thank you for the answer.
>>
>> On Tuesday, November 11, 2014 3:13:18 PM UTC+1, Carl Meyer wrote:
>>
>> Hi David,
>>
>> On 11/11/2014 07:01 AM, dpalao...@gmail.com <javascript:> wrote:
>> > I see your point. You might be right, but it is not clear to me
>> how to
>> > do it and if it would work: I have already tried to subclass
>> > DiscoverRunner to modify its behaviour with little success.
>>
>> If there are specific aspects of "how to do it" that are confusing to
>> you, I could try to clarify. Or if you try it and have specific
>> problems.
>>
>>
>> Well, I never worked with sql directly. And as I said in my post when I
>> tried to subclass DiscoverRunner I got the impression that it is not
>> straightforward (at least for my limited experience with Django).
>>
>> > Another problem that I see: it is not an homogenous approach. I mean,
>> > the models are created from the production database. Now I create an
>> > independent database for testing. Of course, you will tell me that I
>> > have to follow the TDD approach to the end: for the code that creates
>> > the testing database too! And I would agree. But it is clearly a
>> bit too
>> > complicated.
>>
>> I'm afraid I am quite confused by this paragraph.
>>
>> I don't know what you mean by "not an homogenous approach".
>>
>> And I don't know what you mean by "the models are created from the
>> production database."
>>
>> And I don't know what you mean by "an independent database for
>> testing."
>> Django already creates a new database for each test run. I'm not
>> suggesting creating any additional database beyond that, just running
>> some SQL to create your legacy (un-managed) tables in the testing
>> database.
>>
>> The code that creates the testing database will be exercised at the
>> start of every test run, and if it's not working your tests will fail
>> because the legacy tables are not created. Personally, I'd consider
>> that
>> adequate; I wouldn't write any additional tests for that code.
>>
>>
>> I created the models by running "inspectdb". I see that as a (logical)
>> link between the models and the tables. Perhaps not the standard link,
>> but they are related. That is what I meant by "the models are created
>> from the production database".
>
> I see, that makes sense.
>
>> Now your suggestion is to create some tables from scratch. By hand. This
>> is what I mean by "not an homogeneous approach", because I don't see a
>> logical link between the models and the testing database. The testing
>> database is created by me. (Or by Django, then I take control to create
>> the tables, then I return control to Django -- if this is what you suggest).
>
> Sorry, I didn't fully explain this part of my suggestion. I would not
> create the SQL for those tables "by hand" - I would do it via a dump of
> the SQL schema from the actual production tables.
>
>> But actually I think your solution makes a lot of sense. At the
>> beginning I wanted Django to create the testing database along with the
>> tables automatically. And you probably are right and the SQL code to
>> create the tables is absolutely trivial, but for me it is not.
>> So I tried also something slightly different. As the database already
>> exists, I think it would make sense if I also the testing database
>> exists prior to testing.
>> What I just did was:
>> I created the testing database simply with a command like
>>
>> mysqldump -u user --password='xxx' -h localhost -d production_db | mysql
>> -h localhost -u user --passwor='xxx' -Dtest_db
>
> So this is close to my suggestion. But rather than creating the entire
> DB (including managed models too), I would send the dump to a SQL file
> instead:
>
> mysqldump -u user --password='xxx' -h localhost -d production_db >
> schema.sql
>
> And then edit schema.sql to remove any tables for normal managed models,
> leaving only the table definitions for the unmanaged (legacy) tables.
> (You might also add `--no-data`, depending whether you want your legacy
> tables to also be prepopulated with data for your tests).
>
> Then you can allow Django to create its test database normally, and run
> migrations to create tables for your managed models; you just have to
> additionally run `schema.sql` to add in the un-managed tables.
>
> (It's possible that your approach of pre-creating the entire test DB
> could just work, with no need to subclass DiscoverRunner at all, by
> using the --keepdb option to manage.py test; but that option only exists
> in the Django development version, so you probably don't have it.)
>
>> Again, I must subclassing DiscoverRunner to have a chance to succeed
>> (need to by-pass db creation), and actually Django complains if I don't
>> do that.
>> So, I subclassed the DiscoverRunner like this
>>
>> class SkipDBCreationDiscoverRunner(DiscoverRunner):
>> def setup_databases(self, **kwargs):
>> return
>>
>> def teardown_databases(self, old_config, **kwargs):
>> pass
>
> Rather than bypassing database setup and teardown entirely (which will
> cause Django to just use the production database, as you noticed), I
> would override only setup_databases, and still call the super method,
> but just add code after it to run your SQL file and add the table
> definitions for the legacy tables. Something more like this (untested):
>
> from django.db import connection
>
> class MyDiscoverRunner(DiscoverRunner):
> def setup_databases(self, *a, **kw):
> ret = super(MyDiscoverRunner, self).setup_databases(*a, **kw)
> cur = connection.cursor()
> cur.execute(open('schema.sql').read())
> return ret
>
>> and enabled this new thing to run instead of the default in settings.py:
>>
>> TEST_RUNNER="tests.nodb_runner.SkipDBCreationDiscoverRunner"
>>
>> I created a "TEST" entry in the DATABASES dictionary with the proper db
>> name:
>> DATABASES = {
>> 'default': {
>> 'ENGINE': 'mysql.connector.django',
>> 'NAME': 'production_db',
>> 'USER': 'user',
>> 'PASSWORD': 'xxx',
>> 'HOST': 'localhost',
>> 'OPTIONS': {
>> 'autocommit': True,
>> },
>> },
>>
>> 'TEST': {
>> 'ENGINE': 'mysql.connector.django',
>> 'NAME': 'test_db',
>> 'USER': 'user',
>> 'PASSWORD': 'xxx',
>> 'HOST': 'localhost',
>> 'OPTIONS': {
>> 'autocommit': True,
>> },
>> },
>> }
>
> This is not the right format for configurating a test database. What
> you've configured here is an ordinary non-test database that happens to
> be named "TEST" - Django won't do anything special with that. To
> configure a test version of a particular database, you use a 'TEST'
> sub-key, like this (and you don't have to repeat any configuration that
> is the same):
>
> DATABASES = {
> 'default': {
> 'ENGINE': 'mysql.connector.django',
> 'NAME': 'production_db',
> 'USER': 'user',
> 'PASSWORD': 'xxx',
> 'HOST': 'localhost',
> 'OPTIONS': {
> 'autocommit': True,
> },
> 'TEST': {
> 'NAME': 'test_db',
> },
> },
> }
>
>> BUT, Django is using the production database! I don't know what I'm
>> missing. Probably something stupid...
>>
>>
>> > I'm having serious doubts on how friendly Django-1.7 is with
>> respect to
>> > TDD...
>>
>> That seems an unfair conclusion, since your problems here mostly arise
>> from using legacy tables un-managed by Django. That's not the typical
>> situation for a Django project, and it's naturally one that will
>> require
>> some manual work on your part for the corresponding testing setup.
>>
>> Carl
>>
>>
>> Sorry if that sounded rude or disrespectful. It was not my intention. I
>> thought that using Django to work with existing databases was a common
>> case. For me, up to now it has been a bit painful.
>
> This raises the question of whether you really need to be using
> un-managed models at all. The other option is to use normal managed
> models, but just fake the initial (table-creating) migration for those
> models on your production database. Then testing will just work normally.
>
> Carl
>
Dear Carl,
It works!!!
I just made some small changes (see below).
Thanks a lot for your patience and help.
Best,
David
PS The other solution, namely "fake the initial (table-creating)
migration for those models on your production database" is
interesting.
I don't know what solution is better. It feels safer, from my point of
view, playing around with testing databases than with production
databases. But I could consider the other solution if there are strong
enough arguments in favour of it.
---
I summarize the steps I gave to solve it, just for future reference:
Setup: Django.-1.7, Python-3.3
Problem: unit tests crash due to non-existing tables for unmanaged models
Solution: subclass DiscoverRunner to include "by-hand" creation of
tables for unmanaged models.
Details:
1. Create models from tables with "python manage.py inspectdb >
myapp/models.py"
• Edit the file conveniently:
• remove models that are related to Django (they were present
after some tests)
• un-manage the models that represent tables in the production db
2. Create a dump (to file) of the production database structure with
only the un-managed tables in it:
a. mysqldump -u root --password='xxx' -h localhost -d
production_db > tests/production_db_schema.sql
b. Edit "production_db_schema.sql"
1. to remove managed tables, if present.
2. and to join mulitline sql commands in one line.
3. Subclass DiscoverRunner. In my case
a. create a file called "tests/mytest_db_runner.py" inside
the project directory
b. create a subclass of DiscoverRunner in it:
from django.test.runner import DiscoverRunner
from django.conf import settings
from django.db import connection
class NonManagedModelsDiscoverRunner(DiscoverRunner):
"""
DiscoverRunner subclass that creates un-managed tables from a file
containing
the database schema (given by settings.TEST_DB_UNMANAGED_TABLES_SCHEMA_FILE)
"""
def setup_databases(self, **kwargs):
ret = super(NonManagedModelsDiscoverRunner,
self).setup_databases(**kwargs)
cur = connection.cursor()
schema_fn = settings.TEST_DB_UNMANAGED_TABLES_SCHEMA_FILE
with open(schema_fn, "r") as f:
for line in f:
if line.strip() != "":
cur.execute(line)
return ret
c. be sure that the project "settings.py" file contains
something like:
'''
TEST_RUNNER="tests.mytest_db_runner.NonManagedModelsDiscoverRunner"
TEST_DB_UNMANAGED_TABLES_SCHEMA_FILE=os.path.join(BASE_DIR,
"tests/production_db_schema.sql")
'''
4. Configure the project "settings.py" to set a name for the testing
db (if needed, not in my case).
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAKUKWznGh6zFLJV8X8_yBiBgOx6M-jyYO%2BfgLWN2p49_vj7i0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
