Dear Carl,
On Tuesday, February 2, 2016 at 6:23:34 PM UTC, Carl Meyer wrote:
>
> For this scenario, I would use a different settings file (which can all
> "inherit" the same common settings via the `from common_settings import
> *` trick) for each "customer", with a different default database defined
> in each. In fact, is there any reason for one process to have access to
> another process' database at all, in your scenario? I don't think you
> even have a use-case for Django's multi-db support here at all, just a
> use case for multiple settings file variants. Then you can use the
> --settings option with management commands to choose which settings to
> use (or the DJANGO_SETTINGS_MODULE env var where that's convenient); you
> don't need to define your own custom option.
You're right, of course, this is a viable option.
Django's multi-db support is designed for what you're calling "run-time
routing," where a single process needs to access multiple different
> databases at different times. That's not your situation; I'm not sure
> you even need or want multi-db.
>
No, we were looking at it as a possible convenient way of configuring
things, but then again, perhaps not ;-)
> The "design flaw" you are observing here is not specific to databases,
> but to Django settings. Django settings in general are not designed to
> be modified at runtime; they define a static configuration for a given
> process. Some settings can in practice safely be modified on-the-fly
> (but still probably shouldn't be), some cannot, because their value is
> cached for efficiency or other reasons: in general modifying settings at
> runtime is not a supported technique. And it's not a _necessary_
> technique either, in your case, when you could just instead use the
> right settings for each process from the beginning.
>
Yes, but a management command run *is* a process run, and having read
documentation and examples saying that you can pass --data to commands to
operate on a particular database, it doesn't seem to be working as
expected/advertised. Obviously there are many ways to skin this particular
cat, including using multiple setting files, which we of course do for
things like dev vs. test vs. production etc. The idea was that we could use
a special settings file just for populating customer databases, with a
common set of settings ... of course we can use the usual technique of
"inheriting" settings from other modules, and so on.
> I suppose I
> > was expecting transaction.atomic(using=XXX) to not just open a
> > transaction on that database, but also make it the default for the scope
> > of the block (except if explicitly overridden by using() for particular
> > models in code called from that block). The current behaviour seems to
> > violate the principle of least surprise (this my first encounter with
> > multiple databases).
>
> I think Django's design in this case is superior to the one you propose,
> because it keeps two concepts orthogonal that don't need to be linked
> (transactions and multi-db-routing) and avoids implicit global or
> thread-local state changes in operations that don't obviously imply such
> changes, making the overall system more flexible and predictable.
>
> I suppose we'd need a more scientific survey to establish which behavior
> is more surprising to more people :-)
>
Note that my post is about management commands, specifically. In a multi-DB
situation, there could well be times when you want specific management
commands to run against different databases each time they're invoked. What
you are saying is that this can't be done using a --data argument to the
command, where there are typically no issues of thread-locality or the need
to dynamically route to DBs based on HTTP requests or models. Not a very
common use case, to be sure, but not especially esoteric, either. It seems
there is no way to do this in the most natural-seeming way (--data) with
Django out-of-the-box - if I understand you correctly, you would have to
set up multiple settings files, one for each database, where the *only*
difference would the the value of DATABASES['default'] in that settings
file from all the other settings files created for this purpose, and then
invoke that using DJANGO_SETTINGS_MODULE= the particular settings file for
database you wanted to operate on for that run. Possible to do, of course,
but it feels like a workaround rather than a natural approach.
> Is there really no case for an in_database()
> > context manager in Django itself?
>
> Perhaps. In my experience the more typical uses for multi-db are
> amenable to other types of routing (e.g. models X, Y are always routed
> to database A, model Z is always routed to database B, or more complex
> schemes), and dynamically-scoped routing (such as that provided by
> `in_database`) isn't as commonly needed.
>
Agree, this is not a very common use case,
Django provides the database router abstraction, which is a very
> flexible system for implementing whatever kind of multi-db routing you
> need. The burden of proof is heavy on any particular routing scheme to
> demonstrate that it is so frequently needed that it should be bundled
> with Django itself, rather than being a separate reusable package for
> those who need it.
>
I'm not arguing for any particular different routing scheme to be included
- only for management commands to be able to be written to respect --data
arguments passed to them, and which can easily treat the passed value as
the default database to use just for that command invocation, when that
value is different to whatever settings.DATABASES['default'] is. A quick
look at the builtin management commands shows there's a lot of usage of
connections[options.get('database')] going on, and yet externally written
management commands aren't encouraged to use the same approach, and
multiple settings files are the suggested alternative?
Regards,
Vinay Sajip
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-users+unsubscr...@googlegroups.com.
To post to this group, send email to django-users@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/887c16ac-e226-42a1-a102-4b48407ddad0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
